-- Logs begin at Wed 2022-06-15 18:52:06 UTC, end at Wed 2022-06-15 19:59:22 UTC. -- Jun 15 19:23:34.024579 dv-cinder-844563-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 15 19:23:34.031748 dv-cinder-844563-3-iscsi devstack@keystone.service[80215]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 15 19:23:34.033739 dv-cinder-844563-3-iscsi devstack@keystone.service[80215]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jun 15 19:23:34 2022] *** Jun 15 19:23:34.033739 dv-cinder-844563-3-iscsi devstack@keystone.service[80215]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 15 19:23:34.033739 dv-cinder-844563-3-iscsi devstack@keystone.service[80215]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 15 19:23:34.033739 dv-cinder-844563-3-iscsi devstack@keystone.service[80215]: nodename: dv-cinder-844563-3-iscsi Jun 15 19:23:34.033739 dv-cinder-844563-3-iscsi devstack@keystone.service[80215]: machine: x86_64 Jun 15 19:23:34.033739 dv-cinder-844563-3-iscsi devstack@keystone.service[80215]: clock source: unix Jun 15 19:23:34.033739 dv-cinder-844563-3-iscsi devstack@keystone.service[80215]: pcre jit disabled Jun 15 19:23:34.034201 dv-cinder-844563-3-iscsi devstack@keystone.service[80215]: detected number of CPU cores: 4 Jun 15 19:23:34.034201 dv-cinder-844563-3-iscsi devstack@keystone.service[80215]: current working directory: / Jun 15 19:23:34.034201 dv-cinder-844563-3-iscsi devstack@keystone.service[80215]: detected binary path: /usr/bin/uwsgi-core Jun 15 19:23:34.034201 dv-cinder-844563-3-iscsi devstack@keystone.service[80215]: your processes number limit is 47680 Jun 15 19:23:34.034201 dv-cinder-844563-3-iscsi devstack@keystone.service[80215]: your memory page size is 4096 bytes Jun 15 19:23:34.034201 dv-cinder-844563-3-iscsi devstack@keystone.service[80215]: detected max file descriptor number: 2048 Jun 15 19:23:34.034201 dv-cinder-844563-3-iscsi devstack@keystone.service[80215]: lock engine: pthread robust mutexes Jun 15 19:23:34.034201 dv-cinder-844563-3-iscsi devstack@keystone.service[80215]: thunder lock: enabled Jun 15 19:23:34.037451 dv-cinder-844563-3-iscsi devstack@keystone.service[80215]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 15 19:23:34.037451 dv-cinder-844563-3-iscsi devstack@keystone.service[80215]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 15 19:23:34.079061 dv-cinder-844563-3-iscsi devstack@keystone.service[80215]: Python main interpreter initialized at 0x55992fbb19c0 Jun 15 19:23:34.079061 dv-cinder-844563-3-iscsi devstack@keystone.service[80215]: python threads support enabled Jun 15 19:23:34.079061 dv-cinder-844563-3-iscsi devstack@keystone.service[80215]: your server socket listen backlog is limited to 100 connections Jun 15 19:23:34.079061 dv-cinder-844563-3-iscsi devstack@keystone.service[80215]: your mercy for graceful operations on workers is 90 seconds Jun 15 19:23:34.079414 dv-cinder-844563-3-iscsi devstack@keystone.service[80215]: mapped 403077 bytes (393 KB) for 2 cores Jun 15 19:23:34.079523 dv-cinder-844563-3-iscsi devstack@keystone.service[80215]: *** Operational MODE: preforking *** Jun 15 19:23:34.079793 dv-cinder-844563-3-iscsi devstack@keystone.service[80215]: *** uWSGI is running in multiple interpreter mode *** Jun 15 19:23:34.079793 dv-cinder-844563-3-iscsi devstack@keystone.service[80215]: spawned uWSGI master process (pid: 80215) Jun 15 19:23:34.079992 dv-cinder-844563-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 15 19:23:34.080834 dv-cinder-844563-3-iscsi devstack@keystone.service[80215]: spawned uWSGI worker 1 (pid: 80222, cores: 1) Jun 15 19:23:34.080928 dv-cinder-844563-3-iscsi devstack@keystone.service[80215]: spawned uWSGI worker 2 (pid: 80223, cores: 1) Jun 15 19:23:34.082289 dv-cinder-844563-3-iscsi devstack@keystone.service[80215]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 15 19:23:39.595281 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 15 19:23:39.595281 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: from cryptography.utils import int_from_bytes Jun 15 19:23:39.596039 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 15 19:23:39.596039 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: from cryptography.utils import int_from_bytes Jun 15 19:23:39.608590 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 15 19:23:39.608590 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: from cryptography.utils import int_from_bytes Jun 15 19:23:39.609055 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 15 19:23:39.609055 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: from cryptography.utils import int_from_bytes Jun 15 19:23:39.645052 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 15 19:23:39.645242 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 15 19:23:39.645432 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] command line args: [] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 15 19:23:39.645702 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 15 19:23:39.645879 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ================================================================================ {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 15 19:23:39.646161 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] admin_token = **** {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.646411 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] config_dir = [] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.646597 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.646777 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] config_source = [] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.646957 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] debug = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.647154 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.647664 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.647756 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.647951 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.648200 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.648431 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] list_limit = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.648610 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] log_config_append = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.648786 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.649005 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] log_dir = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.649128 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] log_file = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.649364 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.649619 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.649865 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.650131 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.650324 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.650473 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.650721 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.650966 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.651201 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.651434 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.651667 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.651898 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.652147 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.652391 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.652666 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.653151 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.653331 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] publish_errors = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.653563 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.653794 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.653971 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.654252 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.654517 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.654751 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] standard_threads = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.654980 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.655222 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.655397 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.655571 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] use_journal = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.655746 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] use_json = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.655918 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] use_stderr = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.656092 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] use_syslog = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.656177 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 15 19:23:39.656321 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.656409 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 15 19:23:39.656537 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] command line args: [] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 15 19:23:39.656620 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.656801 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.656889 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 15 19:23:39.656961 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ================================================================================ {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 15 19:23:39.657059 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.657214 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] admin_token = **** {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.657291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.657464 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] config_dir = [] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.657552 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.657652 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.657754 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.657841 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] config_source = [] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.657999 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.658082 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] debug = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.658235 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.658344 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.658472 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.658556 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.658654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.658740 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.658905 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.658988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.659153 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.659230 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.659389 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] list_limit = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.659474 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.659569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] log_config_append = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.659662 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.659741 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.659870 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.659955 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] log_dir = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.660072 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] log_file = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.660148 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.660307 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.660391 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.660556 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.660637 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.660801 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.660879 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.661047 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.661126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.661224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.661309 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.661401 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.661517 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.661644 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.661778 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.661889 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.662028 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.662136 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.662301 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.662380 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.662531 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.662611 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.662761 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.662840 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.662989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.663067 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.663223 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.663302 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.663469 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.663549 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.663715 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.663943 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.664027 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.664171 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.664249 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] publish_errors = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.664405 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.664497 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.664635 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.664721 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.664799 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.664871 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.665048 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.665134 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.665284 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.665363 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.665520 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] standard_threads = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.665598 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.665752 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.665907 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.665981 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.666050 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.666185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.666263 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.666363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] use_journal = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.666453 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.666532 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] use_json = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.666648 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.666731 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] use_stderr = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.666865 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] use_syslog = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.666941 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.667097 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 19:23:39.667177 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.667332 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.667409 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.667568 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.667645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.667796 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.667875 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.668030 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.668109 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.668263 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.668341 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.668508 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.668596 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.668737 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.668821 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.668973 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.669058 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.669208 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.669293 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.669444 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.669521 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.669679 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.669758 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.669914 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.670037 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.670191 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.670273 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.670435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.670522 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.670681 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.670758 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.670918 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.670996 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.671147 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.671308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.671391 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.671468 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.671626 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.671704 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.671856 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.671935 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.672092 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.672178 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.672329 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.672407 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.672593 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.672680 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.672827 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.672906 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.673075 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.673153 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.673305 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.673390 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.673538 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.673622 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.673768 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.673871 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.674001 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.674168 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.674273 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.674366 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.674593 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.674692 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.674798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.674883 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.674956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.675082 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.675197 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.675273 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.675388 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.675472 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.675623 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.675706 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.675858 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.675934 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.676091 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.676179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.676340 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.676419 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.676574 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.676660 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.676803 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.676918 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.677018 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.677136 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.677251 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.677331 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.677480 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.677560 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.677717 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.677800 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.677951 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.678030 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.678209 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.678306 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.678437 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.678517 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.678665 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.678789 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.678897 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.678976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.679126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.679206 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.679365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.679454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.679593 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.679832 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.680073 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.680309 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.680546 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.680782 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.681011 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.681088 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.681260 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.681340 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.681496 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.681576 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.681734 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.681813 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.681964 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.682045 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.682245 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.682328 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.682482 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.682563 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] auth.external = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.682714 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.682794 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.682951 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.683070 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.683187 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.683310 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.683467 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.683543 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] auth.password = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.683679 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.683757 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] auth.token = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.683927 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.684011 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.684172 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.684266 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.684408 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.684488 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.684645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.684724 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.684880 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.684959 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.685134 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.685214 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.685371 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.685454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.686143 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.686247 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] credential.caching = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.686397 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.686476 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.686637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.686716 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.686876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.686964 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.687114 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.687200 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.687352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.687437 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.687587 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.687674 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.687848 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.687927 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.688090 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.688175 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.688331 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.688440 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.688570 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.688659 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.688815 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.688896 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.689053 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.689133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.689290 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.689377 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.689528 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.689607 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.689762 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.689849 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.689992 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.690080 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] federation.caching = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.690253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.690333 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.690488 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.690568 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.690718 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.690809 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.690956 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.691036 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.691202 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.691282 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.691438 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.691524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.691678 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.691756 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.691918 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.691996 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.692174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.692415 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.692662 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.692897 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.caching = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.693144 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.693390 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.693602 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.693692 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.693850 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.693929 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.694111 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.694195 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] auth.external = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.694351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.694447 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.694590 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.694675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.694837 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.694916 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.695090 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.695178 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] auth.password = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.695321 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.695400 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] auth.token = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.695551 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.695630 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.695780 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.695868 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.696024 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.696109 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.696267 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.696349 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.696513 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.696600 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.696762 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.696840 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.696997 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.697076 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.697235 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] credential.caching = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.697323 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.697485 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.697565 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.697720 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.697798 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.697951 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.698038 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.698211 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.698293 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.698444 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.698524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.698679 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.698830 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.698906 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.698979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.699147 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.699233 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.699395 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.699477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.699625 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.699705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.699872 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.699950 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.700119 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.700199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.700357 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.700436 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.700605 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.700685 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.700852 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.700933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.701082 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.701161 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] federation.caching = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.701381 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.701967 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.702145 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.702491 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.702588 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.702665 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.702737 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.702809 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.702888 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.702962 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.703065 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.703151 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.703302 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.703382 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.703550 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.703631 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.703796 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.703875 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.704026 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.704105 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.704255 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.704336 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.704500 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.704580 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] identity.caching = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.704748 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.704826 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.704983 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.705062 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.705217 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.705297 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.705452 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.705531 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.705683 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.705764 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.705919 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.705996 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.706184 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.706266 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.706417 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.706497 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.706664 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.706751 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.706898 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.706976 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.707132 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.707214 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.707364 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.707442 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.707593 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.707673 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.707828 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.707908 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.708060 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.708140 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.708296 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.708375 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.708527 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.708608 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.708773 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.708858 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.709018 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.709098 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.709261 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.709340 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.709499 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.709578 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.709739 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.709818 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.709968 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.710048 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.710224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.710304 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.710464 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.710543 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.710702 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.710783 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.710939 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.711018 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.711168 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.711253 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.711412 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.711491 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.711657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.711736 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.711891 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.711977 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.712138 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.712218 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.712366 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.712444 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.712603 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.712689 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.712836 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.712915 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.713064 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.713142 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.713298 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.713384 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.713533 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.713611 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.713772 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.713850 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.714003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.714109 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.714263 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.714342 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.714504 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] resource.caching = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.714582 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.714750 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.714843 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.714981 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.715062 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.715212 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.715291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.715458 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.715544 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.715703 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.715782 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.715936 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.716014 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.716172 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.716259 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.716407 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.716486 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.716635 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.716713 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.716889 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] role.caching = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.716978 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.717140 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] role.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.717221 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.717371 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.717450 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.717620 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.717706 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.717863 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.717942 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.718125 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.718209 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.718367 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.718457 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.718599 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.718679 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.718836 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.718916 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.719074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.719161 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.719313 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.719392 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.719544 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.719631 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.719788 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.719883 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.720031 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.720110 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.720276 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.720354 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.720520 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.720606 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.720765 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.720913 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.720992 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.721152 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.721232 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.721401 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.721483 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.721651 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.721730 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.721899 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.722053 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.722282 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.722357 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.722430 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.722501 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.722624 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.722717 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.722864 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.722944 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.723100 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.723179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.723332 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.723419 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.723565 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.723644 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.723796 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.723875 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.724028 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.724114 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.724268 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.724348 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.724517 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.724596 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.724751 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.724836 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.724981 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.725059 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.725208 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.725286 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.725442 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] token.caching = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.725530 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] resource.caching = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.725680 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.725759 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.725927 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.726006 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.726183 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.726280 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.726435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.726513 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.726677 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.726755 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.726911 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.726996 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.727150 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.727228 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.727386 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.727463 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.727625 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.727720 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.727859 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.727937 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] role.caching = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.728095 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.728181 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] role.driver = sql {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.728326 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.728412 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.728567 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.728645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.728813 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.728892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.729040 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.729125 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.729280 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.729359 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.729538 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.729616 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.729786 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.729873 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.connection = **** {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.730033 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.730138 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.730304 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.730383 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.730535 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.730613 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.730774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.730854 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.731011 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.731090 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.731254 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.731332 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.731488 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.731567 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.731725 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.731880 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.731959 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.732119 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.732209 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.732369 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.732467 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.732643 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.732723 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.732892 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.732980 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.733131 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.733210 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.733366 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.733444 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.733593 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.733699 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.733832 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.733912 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.734066 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.734208 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.734282 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 15 19:23:39.734367 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.734555 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.734787 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.735042 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.735297 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.735547 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.735787 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.736024 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.736255 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.736491 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] token.caching = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.736730 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.736978 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.737214 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.737470 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.737718 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.737955 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.738224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.738470 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.738717 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.738956 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.739196 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.739437 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.739684 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.739933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.740163 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.740406 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.740722 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.740991 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.connection = **** {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.741241 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.741491 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.741731 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.741975 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.742246 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.742488 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.742725 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.742964 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.743210 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.743449 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.743698 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.743935 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.744186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.744424 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.744671 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.744902 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.745144 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.745381 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 19:23:39.745552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80223) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 15 19:23:39.851976 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:39.852150 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:39.852349 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:39.861468 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:39.870559 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:39.870790 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:39.871011 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:39.880059 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:39.889850 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:39.890080 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:39.901977 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:39.902232 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:39.946239 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:39.947452 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:39.994581 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:39.994813 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:39.995372 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:39.995608 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.060606 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.060780 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.060855 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.060999 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.061106 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.061219 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.061305 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.061437 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.061520 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.061651 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.061744 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.061863 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.084925 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.085156 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.085241 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.085381 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.085466 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.085602 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.085685 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.085819 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.085911 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.086031 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.086130 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.086269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.086350 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.086481 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.097741 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.098000 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.110412 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.110589 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.110773 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.110874 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.111003 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.111087 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.111215 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.111297 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.111428 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.111512 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.111639 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.111722 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.111850 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.112061 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80223) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 19:23:40.128991 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.129188 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.129280 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 15 19:23:40.129397 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 15 19:23:40.129522 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.129719 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 15 19:23:40.129823 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.129901 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 15 19:23:40.130183 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.130375 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.130471 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.130638 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.130720 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.130898 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.130979 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.131154 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.142707 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.142897 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.142982 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.143171 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.143254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.143440 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.143524 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.143715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.151606 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 19:23:40.151873 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80223) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 19:23:40.155592 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 19:23:40.155857 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80223) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 19:23:40.155941 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.156133 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.156216 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.156407 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.156498 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.156683 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.156764 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.156938 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.157018 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.157202 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.157285 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.157468 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.157551 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.157731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.157811 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.157994 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.158075 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.158284 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.158365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.158551 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.184634 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.185081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.186302 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 19:23:40.186578 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.186751 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80223) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 19:23:40.187031 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.191836 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 19:23:40.192113 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.192319 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80223) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 19:23:40.192402 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.192594 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.192871 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.199826 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 19:23:40.200105 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.200341 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80223) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 19:23:40.200622 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.205122 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 19:23:40.205407 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.205656 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80223) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 19:23:40.205740 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.205938 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.206037 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.206230 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.206313 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.206498 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.206580 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.206764 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.206858 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.207026 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.207108 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.207295 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.207560 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.226296 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.226572 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.226855 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.227116 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.231149 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 19:23:40.231717 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80223) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 19:23:40.234861 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.235151 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.235455 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.235748 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.240290 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 19:23:40.240902 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80223) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 19:23:40.243938 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 19:23:40.244605 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80223) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 19:23:40.247641 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.247938 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.248757 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.248920 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.249048 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.249219 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.249354 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.249485 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.249616 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.249769 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.250043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.250349 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.268997 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 19:23:40.269280 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.269547 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.269812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80223) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 19:23:40.269898 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.270118 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.270380 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.270648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.277732 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.278586 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.279993 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.280265 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.280846 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.281120 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.283360 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 19:23:40.283651 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.284233 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80223) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 19:23:40.284511 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.285199 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.285510 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.285799 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.286081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.286403 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.286687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.299180 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 19:23:40.299458 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.299746 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.300024 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.300114 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80223) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 19:23:40.300303 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.300394 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.300576 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.300660 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.300851 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.300933 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.301129 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.301402 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.301675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.319383 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 19:23:40.320212 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80223) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 19:23:40.322663 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 19:23:40.323538 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80223) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 19:23:40.325962 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.326935 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.327675 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.328559 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.329472 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 19:23:40.329752 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.330030 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.330374 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80223) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 19:23:40.330653 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.330929 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.337345 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.338247 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.339008 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 19:23:40.339899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80223) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 19:23:40.342812 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.343098 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.343380 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.343655 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.343793 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.343927 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.344207 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.344489 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.353009 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 19:23:40.353293 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.353570 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.353831 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80223) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 19:23:40.354144 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.354423 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.362134 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 19:23:40.362363 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.362641 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.362863 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80223) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 19:23:40.362946 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.363138 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.363222 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.363415 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.363506 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.363687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.363770 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.363963 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.364044 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.364235 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.364319 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.364500 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.364589 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.364768 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.364853 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.365041 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.365123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.365374 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.365463 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.365633 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.365715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.365923 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.366212 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.366479 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80223) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 19:23:40.404222 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55992fbb19c0 pid: 80222 (default app) Jun 15 19:23:40.405135 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55992fbb19c0 pid: 80223 (default app) Jun 15 19:23:40.408245 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88bd52d-f683-485e-9ff8-8d75f0bc171b None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:40.408550 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88bd52d-f683-485e-9ff8-8d75f0bc171b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:40.408829 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88bd52d-f683-485e-9ff8-8d75f0bc171b None None] PATH_INFO: `/v3` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:40.413041 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1/1] 10.222.0.15 () {50 vars in 791 bytes} [Wed Jun 15 19:23:40 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:23:48.121914 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581a1bb-0ae6-42f9-a293-374c48b6de30 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:48.122513 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581a1bb-0ae6-42f9-a293-374c48b6de30 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:48.122513 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581a1bb-0ae6-42f9-a293-374c48b6de30 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:48.127441 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1/2] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:23:48 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 19:23:48.132988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1461a9-2edd-462f-b756-799cc9b9b89a None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:48.133420 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1461a9-2edd-462f-b756-799cc9b9b89a None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:48.133545 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1461a9-2edd-462f-b756-799cc9b9b89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:48.202474 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: INFO dbcounter [None req-0c1461a9-2edd-462f-b756-799cc9b9b89a None None] Registered counter for database keystone Jun 15 19:23:48.223836 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG oslo_db.sqlalchemy.engines [None req-0c1461a9-2edd-462f-b756-799cc9b9b89a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80222) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 15 19:23:48.229943 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writer thread running {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 15 19:23:48.405621 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG passlib.handlers.bcrypt [None req-0c1461a9-2edd-462f-b756-799cc9b9b89a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80222) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 15 19:23:48.405970 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG passlib.handlers.bcrypt [None req-0c1461a9-2edd-462f-b756-799cc9b9b89a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80222) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 15 19:23:48.459338 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-0c1461a9-2edd-462f-b756-799cc9b9b89a None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:23:48.460913 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0c1461a9-2edd-462f-b756-799cc9b9b89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:23:48.580452 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 2/3] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:23:48 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 448 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 15 19:23:48.719591 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b581a1bb-0ae6-42f9-a293-374c48b6de30 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:23:48.789706 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: INFO dbcounter [None req-b581a1bb-0ae6-42f9-a293-374c48b6de30 None None] Registered counter for database keystone Jun 15 19:23:48.809125 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG oslo_db.sqlalchemy.engines [None req-b581a1bb-0ae6-42f9-a293-374c48b6de30 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80223) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 15 19:23:48.815369 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writer thread running {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 15 19:23:48.935287 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b581a1bb-0ae6-42f9-a293-374c48b6de30 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:23:48.939589 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a746c47f-7c68-4bfc-b31b-fea9818dcf60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:23:48.940246 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746c47f-7c68-4bfc-b31b-fea9818dcf60 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:48.940532 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746c47f-7c68-4bfc-b31b-fea9818dcf60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:48.940810 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746c47f-7c68-4bfc-b31b-fea9818dcf60 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:48.948852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a746c47f-7c68-4bfc-b31b-fea9818dcf60 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:23:49.003214 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a746c47f-7c68-4bfc-b31b-fea9818dcf60 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 15 19:23:49.003575 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a746c47f-7c68-4bfc-b31b-fea9818dcf60 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:23:49.014417 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 2/4] 10.222.0.15 () {58 vars in 1324 bytes} [Wed Jun 15 19:23:48 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 19:23:49.020549 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1461a9-2edd-462f-b756-799cc9b9b89a None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:23:49.037634 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1461a9-2edd-462f-b756-799cc9b9b89a None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:23:49.041996 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6402f2-3282-42d5-b277-63c7f4f7db48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:23:49.043129 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6402f2-3282-42d5-b277-63c7f4f7db48 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:49.043415 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6402f2-3282-42d5-b277-63c7f4f7db48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:49.043689 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6402f2-3282-42d5-b277-63c7f4f7db48 admin admin] PATH_INFO: `/v3/projects/7d07be90a27c40718a346a1d9e5469b4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:49.045203 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6402f2-3282-42d5-b277-63c7f4f7db48 admin admin] RBAC: Authorizing `identity:get_project(project_id=7d07be90a27c40718a346a1d9e5469b4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:23:49.095747 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8b6402f2-3282-42d5-b277-63c7f4f7db48 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 15 19:23:49.096088 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6402f2-3282-42d5-b277-63c7f4f7db48 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:23:49.100678 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 3/5] 10.222.0.15 () {56 vars in 1207 bytes} [Wed Jun 15 19:23:49 2022] GET /identity/v3/projects/7d07be90a27c40718a346a1d9e5469b4 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:23:51.166321 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec37be-4a6a-430c-b502-cd5753d471e7 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:51.166859 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec37be-4a6a-430c-b502-cd5753d471e7 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:51.166859 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec37be-4a6a-430c-b502-cd5753d471e7 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:51.172533 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 3/6] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:23:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:23:51.177973 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1358e-8cfd-40ad-8424-b178b2c29194 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:51.178455 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1358e-8cfd-40ad-8424-b178b2c29194 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:51.178543 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1358e-8cfd-40ad-8424-b178b2c29194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:51.198014 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-1ef1358e-8cfd-40ad-8424-b178b2c29194 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:23:51.199558 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1ef1358e-8cfd-40ad-8424-b178b2c29194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:23:51.207299 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 4/7] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:23:51 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 15 19:23:51.229104 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec37be-4a6a-430c-b502-cd5753d471e7 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:23:51.241546 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec37be-4a6a-430c-b502-cd5753d471e7 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:23:51.245807 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675096c-232f-4dc4-9207-68fa718ae6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:23:51.246477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675096c-232f-4dc4-9207-68fa718ae6df admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:51.246768 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675096c-232f-4dc4-9207-68fa718ae6df admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:51.247048 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675096c-232f-4dc4-9207-68fa718ae6df admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:51.255342 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f675096c-232f-4dc4-9207-68fa718ae6df admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:23:51.257123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f675096c-232f-4dc4-9207-68fa718ae6df admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:23:51.266188 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 4/8] 10.222.0.15 () {58 vars in 1324 bytes} [Wed Jun 15 19:23:51 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 19:23:51.271168 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1358e-8cfd-40ad-8424-b178b2c29194 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:23:51.279943 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1358e-8cfd-40ad-8424-b178b2c29194 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:23:51.284209 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a7d3dd-0a0b-430c-ba97-26005a6ba787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:23:51.285065 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7d3dd-0a0b-430c-ba97-26005a6ba787 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:51.285349 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7d3dd-0a0b-430c-ba97-26005a6ba787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:51.285626 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7d3dd-0a0b-430c-ba97-26005a6ba787 admin admin] PATH_INFO: `/v3/users/d018cc34dbeb4d56a3052de5c20b961c` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:51.287032 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a7d3dd-0a0b-430c-ba97-26005a6ba787 admin admin] RBAC: Authorizing `identity:get_user(user_id=d018cc34dbeb4d56a3052de5c20b961c)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:23:51.288830 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a7d3dd-0a0b-430c-ba97-26005a6ba787 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:23:51.290408 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 5/9] 10.222.0.15 () {56 vars in 1198 bytes} [Wed Jun 15 19:23:51 2022] GET /identity/v3/users/d018cc34dbeb4d56a3052de5c20b961c => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:23:53.554670 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a01e5f-a1ed-4c76-af3c-4cd260b9127c None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:53.555242 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a01e5f-a1ed-4c76-af3c-4cd260b9127c None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:53.555242 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a01e5f-a1ed-4c76-af3c-4cd260b9127c None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:53.556657 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 5/10] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:23:53.562701 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2da3d1-a564-4dcf-84fb-feaab0f6a329 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:53.563212 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2da3d1-a564-4dcf-84fb-feaab0f6a329 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:53.563407 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2da3d1-a564-4dcf-84fb-feaab0f6a329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:53.581259 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-bb2da3d1-a564-4dcf-84fb-feaab0f6a329 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:23:53.582503 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-bb2da3d1-a564-4dcf-84fb-feaab0f6a329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:23:53.618854 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef2475-401f-4ecc-917f-31c473a63320 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:53.619070 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef2475-401f-4ecc-917f-31c473a63320 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:53.619342 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef2475-401f-4ecc-917f-31c473a63320 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:53.620712 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 6/11] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 19:23:53.625628 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd573a-056f-4d0e-a40b-6b442d97fddc None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:53.625925 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd573a-056f-4d0e-a40b-6b442d97fddc None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:53.626244 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd573a-056f-4d0e-a40b-6b442d97fddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:53.663380 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 6/12] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:23:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:23:53.666506 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG passlib.handlers.bcrypt [None req-a9fd573a-056f-4d0e-a40b-6b442d97fddc None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80223) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 15 19:23:53.666942 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG passlib.handlers.bcrypt [None req-a9fd573a-056f-4d0e-a40b-6b442d97fddc None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80223) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 15 19:23:53.685278 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2da3d1-a564-4dcf-84fb-feaab0f6a329 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:23:53.708086 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:23:53.712229 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-a9fd573a-056f-4d0e-a40b-6b442d97fddc None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:23:53.713822 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-a9fd573a-056f-4d0e-a40b-6b442d97fddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:23:53.788745 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2da3d1-a564-4dcf-84fb-feaab0f6a329 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:23:53.814275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 7/13] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:23:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:23:53.822247 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d0aa97-9de3-4ed3-bc0f-0b691463160e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:23:53.823046 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0aa97-9de3-4ed3-bc0f-0b691463160e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:53.823334 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0aa97-9de3-4ed3-bc0f-0b691463160e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:53.823615 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0aa97-9de3-4ed3-bc0f-0b691463160e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:53.834730 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d0aa97-9de3-4ed3-bc0f-0b691463160e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:23:53.835805 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd573a-056f-4d0e-a40b-6b442d97fddc None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:23:53.846516 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-06d0aa97-9de3-4ed3-bc0f-0b691463160e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:23:53.846516 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:23:53.846516 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:23:53.846516 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:23:53.846516 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:23:53.846516 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:23:53.846516 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:23:53.846516 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:23:53.846516 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:23:53.846516 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:23:53.846516 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:23:53.846516 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:23:53.846516 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:23:53.846516 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:23:53.846516 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:23:53.846516 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:23:53.847615 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:23:53.847615 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:23:53.847615 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:23:53.847615 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:23:53.847615 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:23:53.847615 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:23:53.847615 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:23:53.847615 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:23:53.847615 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:23:53.847615 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:23:53.847615 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:23:53.847615 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:23:53.847615 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:23:53.847615 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:23:53.847615 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:23:53.847615 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:23:53.848637 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:23:53.848637 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:23:53.848637 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:23:53.848637 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:23:53.848637 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:23:53.848637 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:23:53.848637 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:23:53.848637 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:23:53.848637 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:23:53.848637 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:23:53.848637 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:23:53.848637 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:23:53.848637 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 7/14] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jun 15 19:23:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:23:53.855027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d0aa97-9de3-4ed3-bc0f-0b691463160e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:23:53.934696 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:23:53.935087 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd573a-056f-4d0e-a40b-6b442d97fddc None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:23:53.937575 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:23:53.953867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d0aa97-9de3-4ed3-bc0f-0b691463160e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:23:53.974732 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2266cc93-b709-47c8-801b-5b3647d518a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:23:53.975625 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2266cc93-b709-47c8-801b-5b3647d518a0 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:53.975909 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2266cc93-b709-47c8-801b-5b3647d518a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:53.976184 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2266cc93-b709-47c8-801b-5b3647d518a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:53.988310 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e98bc-ea41-4ca0-bbc6-00ad3dcdb7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:23:53.988975 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e98bc-ea41-4ca0-bbc6-00ad3dcdb7bf None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:53.989262 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e98bc-ea41-4ca0-bbc6-00ad3dcdb7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:53.989542 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e98bc-ea41-4ca0-bbc6-00ad3dcdb7bf None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:53.990328 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e98bc-ea41-4ca0-bbc6-00ad3dcdb7bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:23:53.990642 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e98bc-ea41-4ca0-bbc6-00ad3dcdb7bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:23:53.992037 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e98bc-ea41-4ca0-bbc6-00ad3dcdb7bf None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:23:54.003666 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 8/15] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:23:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:23:54.009631 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e98bc-ea41-4ca0-bbc6-00ad3dcdb7bf None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:23:54.043242 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2266cc93-b709-47c8-801b-5b3647d518a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:23:54.045703 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2266cc93-b709-47c8-801b-5b3647d518a0 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:23:54.108232 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e98bc-ea41-4ca0-bbc6-00ad3dcdb7bf None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:23:54.141655 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 8/16] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:23:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 19:23:54.142037 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeb48c5-d728-4fa6-bc55-ab7cc3da0508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:23:54.142989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb48c5-d728-4fa6-bc55-ab7cc3da0508 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:54.143280 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb48c5-d728-4fa6-bc55-ab7cc3da0508 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:54.143558 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb48c5-d728-4fa6-bc55-ab7cc3da0508 None admin] PATH_INFO: `/v3/users/d018cc34dbeb4d56a3052de5c20b961c` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:54.144793 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeb48c5-d728-4fa6-bc55-ab7cc3da0508 None admin] RBAC: Authorizing `identity:get_user(user_id=d018cc34dbeb4d56a3052de5c20b961c)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:23:54.146373 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeb48c5-d728-4fa6-bc55-ab7cc3da0508 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:23:54.147158 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2266cc93-b709-47c8-801b-5b3647d518a0 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:23:54.147998 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 9/17] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:23:54 2022] GET /identity/v3/users/d018cc34dbeb4d56a3052de5c20b961c => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:23:54.149846 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:23:54.153299 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeb48c5-d728-4fa6-bc55-ab7cc3da0508 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:23:54.168047 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:23:54.248846 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2266cc93-b709-47c8-801b-5b3647d518a0 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:23:54.260665 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeb48c5-d728-4fa6-bc55-ab7cc3da0508 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:23:54.286272 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd214e-1e53-42b1-b5db-f0dea2af3eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:23:54.287066 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd214e-1e53-42b1-b5db-f0dea2af3eed None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:54.287352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd214e-1e53-42b1-b5db-f0dea2af3eed None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:54.287626 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd214e-1e53-42b1-b5db-f0dea2af3eed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:54.295402 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4718958-55ee-4db2-b883-f1d3223ad627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:23:54.296197 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4718958-55ee-4db2-b883-f1d3223ad627 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:54.296483 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4718958-55ee-4db2-b883-f1d3223ad627 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:54.296757 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4718958-55ee-4db2-b883-f1d3223ad627 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:54.297958 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4718958-55ee-4db2-b883-f1d3223ad627 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:23:54.299465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4718958-55ee-4db2-b883-f1d3223ad627 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:23:54.301069 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 10/18] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:23:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:23:54.305896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd214e-1e53-42b1-b5db-f0dea2af3eed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:23:54.306723 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4718958-55ee-4db2-b883-f1d3223ad627 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:23:54.307199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd214e-1e53-42b1-b5db-f0dea2af3eed None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:23:54.328352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-39cd214e-1e53-42b1-b5db-f0dea2af3eed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:23:54.328352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:23:54.328352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:23:54.328352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:23:54.328352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:23:54.328352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:23:54.328352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:23:54.328352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:23:54.328352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:23:54.328352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:23:54.328352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:23:54.328352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:23:54.328352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:23:54.328352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:23:54.328352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:23:54.329211 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:23:54.329211 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:23:54.329211 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:23:54.329211 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:23:54.329211 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:23:54.329211 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:23:54.329211 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:23:54.329211 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:23:54.329211 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:23:54.329211 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:23:54.329211 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:23:54.329211 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:23:54.329211 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:23:54.329211 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:23:54.329211 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:23:54.329211 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:23:54.329211 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:23:54.330016 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:23:54.330016 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:23:54.330016 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:23:54.330016 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:23:54.330016 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 9/19] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:23:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:23:54.335172 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd214e-1e53-42b1-b5db-f0dea2af3eed None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:23:54.390644 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:23:54.393032 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:23:54.412638 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4718958-55ee-4db2-b883-f1d3223ad627 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:23:54.435986 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd214e-1e53-42b1-b5db-f0dea2af3eed None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:23:54.445370 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d854e962-b552-4374-ae5a-b555e4fe07e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:23:54.446043 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854e962-b552-4374-ae5a-b555e4fe07e6 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:54.446358 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854e962-b552-4374-ae5a-b555e4fe07e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:54.446633 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854e962-b552-4374-ae5a-b555e4fe07e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:54.447467 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d854e962-b552-4374-ae5a-b555e4fe07e6 None admin] RBAC: Adding query filter params (role.id=4f736313c26d4f8db48bbb2f346e50e4, scope.domain.id=default, user.id=d018cc34dbeb4d56a3052de5c20b961c) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:23:54.447776 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d854e962-b552-4374-ae5a-b555e4fe07e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:23:54.449199 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d854e962-b552-4374-ae5a-b555e4fe07e6 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:23:54.456337 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 11/20] 10.222.0.15 () {56 vars in 1322 bytes} [Wed Jun 15 19:23:54 2022] GET /identity/v3/role_assignments?user.id=d018cc34dbeb4d56a3052de5c20b961c&scope.domain.id=default&role.id=4f736313c26d4f8db48bbb2f346e50e4 => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:23:54.470325 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20945f44-80be-4389-8a3e-1a2788c2b10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:23:54.471016 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20945f44-80be-4389-8a3e-1a2788c2b10a None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:54.471305 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20945f44-80be-4389-8a3e-1a2788c2b10a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:54.471586 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20945f44-80be-4389-8a3e-1a2788c2b10a None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:54.472338 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20945f44-80be-4389-8a3e-1a2788c2b10a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:23:54.472652 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20945f44-80be-4389-8a3e-1a2788c2b10a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:23:54.474124 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20945f44-80be-4389-8a3e-1a2788c2b10a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:23:54.502008 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 10/21] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:23:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:23:56.758635 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ee900-72f5-4271-b7fe-dea8aa6693b3 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:56.759906 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ee900-72f5-4271-b7fe-dea8aa6693b3 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:56.760387 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ee900-72f5-4271-b7fe-dea8aa6693b3 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:56.761967 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 12/22] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:23:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:23:56.767183 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae87ff-7d28-4acd-8054-e0c1ce415333 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:56.767557 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae87ff-7d28-4acd-8054-e0c1ce415333 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:56.767744 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae87ff-7d28-4acd-8054-e0c1ce415333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:56.791082 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-77ae87ff-7d28-4acd-8054-e0c1ce415333 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:23:56.792623 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-77ae87ff-7d28-4acd-8054-e0c1ce415333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:23:56.882070 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 11/23] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:23:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:23:56.915372 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ee900-72f5-4271-b7fe-dea8aa6693b3 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:23:57.057004 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ee900-72f5-4271-b7fe-dea8aa6693b3 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:23:57.102781 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74b1f2-8da6-45a6-9b9c-a02fd23f9540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:23:57.103971 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74b1f2-8da6-45a6-9b9c-a02fd23f9540 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:57.104444 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74b1f2-8da6-45a6-9b9c-a02fd23f9540 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:57.104906 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74b1f2-8da6-45a6-9b9c-a02fd23f9540 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:57.123501 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74b1f2-8da6-45a6-9b9c-a02fd23f9540 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:23:57.131598 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-da74b1f2-8da6-45a6-9b9c-a02fd23f9540 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:23:57.131598 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:23:57.131598 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:23:57.131598 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:23:57.131598 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:23:57.131598 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:23:57.131598 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:23:57.131598 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:23:57.131598 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:23:57.131598 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:23:57.131598 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:23:57.131598 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:23:57.131598 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:23:57.131598 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:23:57.131598 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:23:57.131598 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:23:57.132465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:23:57.132465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:23:57.132465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:23:57.132465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:23:57.132465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:23:57.132465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:23:57.132465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:23:57.132465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:23:57.132465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:23:57.132465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:23:57.132465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:23:57.132465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:23:57.132465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:23:57.132465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:23:57.132465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:23:57.132465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:23:57.133213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:23:57.133213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:23:57.133213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:23:57.133213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:23:57.133213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:23:57.133213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:23:57.133213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:23:57.133213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:23:57.133213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:23:57.133213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:23:57.133213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:23:57.133213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:23:57.136011 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 13/24] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jun 15 19:23:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:23:57.142703 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae87ff-7d28-4acd-8054-e0c1ce415333 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:23:57.235313 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:23:57.310621 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae87ff-7d28-4acd-8054-e0c1ce415333 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:23:57.316720 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc74b35-3d52-4a00-8fed-414052cca605 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:57.317268 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc74b35-3d52-4a00-8fed-414052cca605 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:57.317758 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc74b35-3d52-4a00-8fed-414052cca605 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:57.319920 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 14/25] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:23:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 19:23:57.335743 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60563da5-77db-43d4-92bf-078db5651163 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:57.336357 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60563da5-77db-43d4-92bf-078db5651163 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:57.336826 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60563da5-77db-43d4-92bf-078db5651163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:57.358762 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:23:57.380889 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-60563da5-77db-43d4-92bf-078db5651163 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:23:57.382117 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-60563da5-77db-43d4-92bf-078db5651163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:23:57.411227 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4aaf67-5340-43e5-bb79-3ef66f67bff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:23:57.411918 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4aaf67-5340-43e5-bb79-3ef66f67bff0 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:57.412204 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4aaf67-5340-43e5-bb79-3ef66f67bff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:57.412482 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4aaf67-5340-43e5-bb79-3ef66f67bff0 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:57.413246 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4aaf67-5340-43e5-bb79-3ef66f67bff0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:23:57.413554 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4aaf67-5340-43e5-bb79-3ef66f67bff0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:23:57.414975 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4aaf67-5340-43e5-bb79-3ef66f67bff0 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:23:57.455824 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 12/26] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:23:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:23:57.459945 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4aaf67-5340-43e5-bb79-3ef66f67bff0 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:23:57.590520 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 15/27] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:23:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 19:23:57.666863 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4aaf67-5340-43e5-bb79-3ef66f67bff0 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:23:57.668987 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60563da5-77db-43d4-92bf-078db5651163 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:23:57.680200 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:23:57.721499 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5661a9a-3a48-4d57-a10c-857a6b7b3ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:23:57.722866 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5661a9a-3a48-4d57-a10c-857a6b7b3ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:57.723351 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5661a9a-3a48-4d57-a10c-857a6b7b3ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:57.723803 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5661a9a-3a48-4d57-a10c-857a6b7b3ad2 None admin] PATH_INFO: `/v3/users/d018cc34dbeb4d56a3052de5c20b961c` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:57.725628 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5661a9a-3a48-4d57-a10c-857a6b7b3ad2 None admin] RBAC: Authorizing `identity:get_user(user_id=d018cc34dbeb4d56a3052de5c20b961c)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:23:57.727455 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5661a9a-3a48-4d57-a10c-857a6b7b3ad2 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:23:57.729360 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 13/28] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:23:57 2022] GET /identity/v3/users/d018cc34dbeb4d56a3052de5c20b961c => generated 247 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:23:57.730956 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ea228-cf6c-46e5-9304-8396490da36e None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:57.731421 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ea228-cf6c-46e5-9304-8396490da36e None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:57.731869 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ea228-cf6c-46e5-9304-8396490da36e None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:57.733548 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 14/29] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:23:57.739280 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1aa58f-66d8-4943-9827-020cf6498d9a None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:57.739782 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1aa58f-66d8-4943-9827-020cf6498d9a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:57.740236 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1aa58f-66d8-4943-9827-020cf6498d9a None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:57.741651 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 15/30] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:23:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:23:57.742574 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1aa58f-66d8-4943-9827-020cf6498d9a None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:23:57.813696 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60563da5-77db-43d4-92bf-078db5651163 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:23:57.853294 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3eb5e-554e-4ec4-937d-9dea9d4ed64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:23:57.853995 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3eb5e-554e-4ec4-937d-9dea9d4ed64e None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:57.854317 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3eb5e-554e-4ec4-937d-9dea9d4ed64e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:57.854645 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3eb5e-554e-4ec4-937d-9dea9d4ed64e None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:57.855532 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3eb5e-554e-4ec4-937d-9dea9d4ed64e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:23:57.856994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3eb5e-554e-4ec4-937d-9dea9d4ed64e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:23:57.859889 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:23:57.867442 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1aa58f-66d8-4943-9827-020cf6498d9a None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:23:57.906883 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca5120-0d8d-4e1a-acde-8a8c9c496b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:23:57.908273 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca5120-0d8d-4e1a-acde-8a8c9c496b46 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:57.908687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca5120-0d8d-4e1a-acde-8a8c9c496b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:57.909044 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca5120-0d8d-4e1a-acde-8a8c9c496b46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:57.910428 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ca5120-0d8d-4e1a-acde-8a8c9c496b46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:23:57.912038 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ca5120-0d8d-4e1a-acde-8a8c9c496b46 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:23:57.914330 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 16/31] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:23:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:23:57.915782 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edde4ac-8edb-4846-a4b0-64e1ad8b2405 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:57.916159 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edde4ac-8edb-4846-a4b0-64e1ad8b2405 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:57.916530 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edde4ac-8edb-4846-a4b0-64e1ad8b2405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:57.934599 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-0edde4ac-8edb-4846-a4b0-64e1ad8b2405 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:23:57.935989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-0edde4ac-8edb-4846-a4b0-64e1ad8b2405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:23:57.980331 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:23:58.018468 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 17/32] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:23:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:23:58.020275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbffe4-a7b1-4737-bea5-8ab7b6bbe8bc None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:58.020634 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbffe4-a7b1-4737-bea5-8ab7b6bbe8bc None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:58.020984 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbffe4-a7b1-4737-bea5-8ab7b6bbe8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:58.052459 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-96bbffe4-a7b1-4737-bea5-8ab7b6bbe8bc None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:23:58.053750 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-96bbffe4-a7b1-4737-bea5-8ab7b6bbe8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:23:58.115220 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 16/33] 10.222.0.15 () {60 vars in 1131 bytes} [Wed Jun 15 19:23:57 2022] POST /identity/v3/roles => generated 219 bytes in 447 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:23:58.116564 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3eb5e-554e-4ec4-937d-9dea9d4ed64e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:23:58.134609 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 18/34] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:23:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:23:58.135574 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bbffe4-a7b1-4737-bea5-8ab7b6bbe8bc None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:23:58.242955 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3eb5e-554e-4ec4-937d-9dea9d4ed64e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:23:58.245566 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bbffe4-a7b1-4737-bea5-8ab7b6bbe8bc None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:23:58.248920 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:23:58.285304 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4933aa89-5f10-46bd-8d37-5f274ac0971b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:23:58.286228 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933aa89-5f10-46bd-8d37-5f274ac0971b None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:58.286593 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933aa89-5f10-46bd-8d37-5f274ac0971b None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:58.286889 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933aa89-5f10-46bd-8d37-5f274ac0971b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:58.288125 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a730675a-f4c5-4ed7-afd0-73845e6a107d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:23:58.289380 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a730675a-f4c5-4ed7-afd0-73845e6a107d None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:58.289877 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a730675a-f4c5-4ed7-afd0-73845e6a107d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:58.290489 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a730675a-f4c5-4ed7-afd0-73845e6a107d None admin] PATH_INFO: `/v3/domains/default/users/d018cc34dbeb4d56a3052de5c20b961c/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:58.293125 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a730675a-f4c5-4ed7-afd0-73845e6a107d None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d018cc34dbeb4d56a3052de5c20b961c, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:23:58.295277 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a730675a-f4c5-4ed7-afd0-73845e6a107d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:23:58.299378 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4933aa89-5f10-46bd-8d37-5f274ac0971b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:23:58.301019 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4933aa89-5f10-46bd-8d37-5f274ac0971b None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:23:58.323170 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-4933aa89-5f10-46bd-8d37-5f274ac0971b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:23:58.323170 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:23:58.323170 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:23:58.323170 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:23:58.323170 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:23:58.323170 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:23:58.323170 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:23:58.323170 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:23:58.323170 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:23:58.323170 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:23:58.323170 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:23:58.323170 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:23:58.323170 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:23:58.323170 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:23:58.323170 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:23:58.324093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:23:58.324093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:23:58.324093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:23:58.324093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:23:58.324093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:23:58.324093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:23:58.324093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:23:58.324093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:23:58.324093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:23:58.324093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:23:58.324093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:23:58.324093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:23:58.324093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:23:58.324093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:23:58.324093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:23:58.324093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:23:58.324093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:23:58.324908 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:23:58.324908 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:23:58.324908 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:23:58.324908 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:23:58.324908 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 19/35] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:23:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:23:58.325653 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4933aa89-5f10-46bd-8d37-5f274ac0971b None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:23:58.347010 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 17/36] 10.222.0.15 () {58 vars in 1361 bytes} [Wed Jun 15 19:23:58 2022] PUT /identity/v3/domains/default/users/d018cc34dbeb4d56a3052de5c20b961c/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:23:58.348106 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a730675a-f4c5-4ed7-afd0-73845e6a107d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:23:58.498686 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4933aa89-5f10-46bd-8d37-5f274ac0971b None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:23:58.514704 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:23:58.564775 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48755c1-e7b8-4ff7-a9ba-7d5751e7fded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:23:58.565466 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48755c1-e7b8-4ff7-a9ba-7d5751e7fded None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:58.565762 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48755c1-e7b8-4ff7-a9ba-7d5751e7fded None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:58.566147 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48755c1-e7b8-4ff7-a9ba-7d5751e7fded None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:58.566854 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48755c1-e7b8-4ff7-a9ba-7d5751e7fded None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:23:58.568628 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48755c1-e7b8-4ff7-a9ba-7d5751e7fded None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:23:58.590567 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a730675a-f4c5-4ed7-afd0-73845e6a107d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:23:58.629175 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d85fe80-36b9-4858-811d-89bba64107bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:23:58.629840 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85fe80-36b9-4858-811d-89bba64107bb None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:58.630149 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85fe80-36b9-4858-811d-89bba64107bb None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:58.630431 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85fe80-36b9-4858-811d-89bba64107bb None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:58.631165 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d85fe80-36b9-4858-811d-89bba64107bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:23:58.631559 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d85fe80-36b9-4858-811d-89bba64107bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:23:58.632944 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d85fe80-36b9-4858-811d-89bba64107bb None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:23:58.652235 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 18/37] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:23:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:23:58.653546 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f59c16-9779-4c20-ae99-d054e697017c None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:58.654468 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f59c16-9779-4c20-ae99-d054e697017c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:58.654760 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f59c16-9779-4c20-ae99-d054e697017c None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:58.656425 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 19/38] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:23:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:23:58.657670 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add0365-e976-4b83-9cbb-8fbc8ceb06af None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:58.657961 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add0365-e976-4b83-9cbb-8fbc8ceb06af None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:58.662033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add0365-e976-4b83-9cbb-8fbc8ceb06af None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:58.664024 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 20/39] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:23:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:23:58.664744 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5add0365-e976-4b83-9cbb-8fbc8ceb06af None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:23:58.775926 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5add0365-e976-4b83-9cbb-8fbc8ceb06af None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:23:58.790972 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:23:58.813060 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5694e727-224d-40d7-96ab-cdb3c106e0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:23:58.813798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694e727-224d-40d7-96ab-cdb3c106e0f6 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:58.814381 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694e727-224d-40d7-96ab-cdb3c106e0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:58.814468 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694e727-224d-40d7-96ab-cdb3c106e0f6 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:58.822607 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 20/40] 10.222.0.15 () {60 vars in 1131 bytes} [Wed Jun 15 19:23:58 2022] POST /identity/v3/roles => generated 225 bytes in 489 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:23:58.822792 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5694e727-224d-40d7-96ab-cdb3c106e0f6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:23:58.822868 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254876c-c255-4d43-a969-718db468b6bd None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:58.822868 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254876c-c255-4d43-a969-718db468b6bd None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:58.823003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5694e727-224d-40d7-96ab-cdb3c106e0f6 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:23:58.823069 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254876c-c255-4d43-a969-718db468b6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:58.842313 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-0254876c-c255-4d43-a969-718db468b6bd None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:23:58.843930 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-0254876c-c255-4d43-a969-718db468b6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:23:58.861296 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:23:58.872163 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 21/41] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Jun 15 19:23:58 2022] POST /identity/v3/projects => generated 301 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:23:58.873874 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be8523-5414-4b96-8871-d77ad8c5f804 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:58.874227 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be8523-5414-4b96-8871-d77ad8c5f804 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:58.875617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be8523-5414-4b96-8871-d77ad8c5f804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:58.899697 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-03be8523-5414-4b96-8871-d77ad8c5f804 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:23:58.901364 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-03be8523-5414-4b96-8871-d77ad8c5f804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:23:58.939362 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 21/42] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:23:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:23:58.982655 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0254876c-c255-4d43-a969-718db468b6bd None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:23:58.999073 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 22/43] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:23:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:23:59.039655 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be8523-5414-4b96-8871-d77ad8c5f804 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:23:59.111505 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0254876c-c255-4d43-a969-718db468b6bd None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:23:59.161559 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd799df-18ee-4ec9-8a15-41608bd4575a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:23:59.162321 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd799df-18ee-4ec9-8a15-41608bd4575a None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:59.162607 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd799df-18ee-4ec9-8a15-41608bd4575a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:59.162884 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd799df-18ee-4ec9-8a15-41608bd4575a None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:59.163659 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd799df-18ee-4ec9-8a15-41608bd4575a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:23:59.165173 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd799df-18ee-4ec9-8a15-41608bd4575a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:23:59.179854 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 22/44] 10.222.0.15 () {60 vars in 1131 bytes} [Wed Jun 15 19:23:58 2022] POST /identity/v3/roles => generated 223 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:23:59.188656 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:23:59.196956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be8523-5414-4b96-8871-d77ad8c5f804 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:23:59.241172 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695045e0-4d21-46df-acb9-8b3376a0619c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:23:59.242373 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695045e0-4d21-46df-acb9-8b3376a0619c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:59.242837 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695045e0-4d21-46df-acb9-8b3376a0619c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:59.243267 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695045e0-4d21-46df-acb9-8b3376a0619c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:59.244972 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695045e0-4d21-46df-acb9-8b3376a0619c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:23:59.246617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695045e0-4d21-46df-acb9-8b3376a0619c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:23:59.248695 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 23/45] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:23:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:23:59.254260 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd799df-18ee-4ec9-8a15-41608bd4575a None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:23:59.264051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:23:59.373712 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd799df-18ee-4ec9-8a15-41608bd4575a None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:23:59.407283 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9978ad-e13c-4c6e-a7ae-b3c81dbe28cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:23:59.407957 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9978ad-e13c-4c6e-a7ae-b3c81dbe28cf None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:23:59.408247 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9978ad-e13c-4c6e-a7ae-b3c81dbe28cf None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:23:59.408538 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9978ad-e13c-4c6e-a7ae-b3c81dbe28cf None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:23:59.409265 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9978ad-e13c-4c6e-a7ae-b3c81dbe28cf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:23:59.410765 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9978ad-e13c-4c6e-a7ae-b3c81dbe28cf None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:23:59.438672 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 23/46] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Jun 15 19:23:59 2022] POST /identity/v3/projects => generated 312 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:24:00.612727 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c9793-46db-41b4-8a3a-021383268d66 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:00.613205 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c9793-46db-41b4-8a3a-021383268d66 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:00.613205 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c9793-46db-41b4-8a3a-021383268d66 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:00.614630 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 24/47] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:00.619708 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625ede6-a1f7-43a5-8823-9d5f776b0ff3 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:00.620077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625ede6-a1f7-43a5-8823-9d5f776b0ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:00.620266 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625ede6-a1f7-43a5-8823-9d5f776b0ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:00.637409 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-f625ede6-a1f7-43a5-8823-9d5f776b0ff3 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:00.639233 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-f625ede6-a1f7-43a5-8823-9d5f776b0ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:00.689890 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:00.704912 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 24/48] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:00.727332 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664c9793-46db-41b4-8a3a-021383268d66 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:00.827517 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664c9793-46db-41b4-8a3a-021383268d66 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:00.860323 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f9a32-86b7-4922-b239-de460538c6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:00.861108 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f9a32-86b7-4922-b239-de460538c6a7 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:00.861397 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f9a32-86b7-4922-b239-de460538c6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:00.861676 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f9a32-86b7-4922-b239-de460538c6a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:00.871647 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593f9a32-86b7-4922-b239-de460538c6a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:00.879878 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-593f9a32-86b7-4922-b239-de460538c6a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:24:00.879878 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:00.879878 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:00.879878 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:00.879878 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:00.879878 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:00.879878 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:00.879878 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:00.879878 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:00.879878 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:00.879878 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:00.879878 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:00.879878 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:00.879878 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:00.879878 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:00.879878 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:00.880774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:00.880774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:00.880774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:00.880774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:00.880774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:00.880774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:00.880774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:00.880774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:00.880774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:00.880774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:00.880774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:00.880774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:00.880774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:00.880774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:00.880774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:00.880774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:00.881631 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:00.881631 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:00.881631 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:00.881631 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:00.881631 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:00.881631 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:00.881631 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:00.881631 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:00.881631 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:00.881631 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:00.881631 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:24:00.881631 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:24:00.881631 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 25/49] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jun 15 19:24:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:00.887005 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625ede6-a1f7-43a5-8823-9d5f776b0ff3 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:00.981780 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625ede6-a1f7-43a5-8823-9d5f776b0ff3 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:01.015952 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b4a86-cbab-48ae-8434-c67eeece8f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:01.016631 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b4a86-cbab-48ae-8434-c67eeece8f77 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:01.016922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b4a86-cbab-48ae-8434-c67eeece8f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:01.017207 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b4a86-cbab-48ae-8434-c67eeece8f77 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:01.017963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b4a86-cbab-48ae-8434-c67eeece8f77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:01.018314 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b4a86-cbab-48ae-8434-c67eeece8f77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:01.019717 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b4a86-cbab-48ae-8434-c67eeece8f77 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:01.027860 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 25/50] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:24:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:01.033904 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f9a32-86b7-4922-b239-de460538c6a7 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:01.059452 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:01.136068 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f9a32-86b7-4922-b239-de460538c6a7 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:01.169487 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab7361-4f3d-4230-8b7d-fba5af6ad6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:01.170386 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab7361-4f3d-4230-8b7d-fba5af6ad6bd None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:01.170678 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab7361-4f3d-4230-8b7d-fba5af6ad6bd None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:01.170961 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab7361-4f3d-4230-8b7d-fba5af6ad6bd None admin] PATH_INFO: `/v3/users/d018cc34dbeb4d56a3052de5c20b961c` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:01.172166 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab7361-4f3d-4230-8b7d-fba5af6ad6bd None admin] RBAC: Authorizing `identity:get_user(user_id=d018cc34dbeb4d56a3052de5c20b961c)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:01.173638 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab7361-4f3d-4230-8b7d-fba5af6ad6bd None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:01.175528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 26/51] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:24:01 2022] GET /identity/v3/users/d018cc34dbeb4d56a3052de5c20b961c => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:01.180636 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b4a86-cbab-48ae-8434-c67eeece8f77 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:01.229858 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:01.283067 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b4a86-cbab-48ae-8434-c67eeece8f77 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:01.315539 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e9726-985c-4236-9686-ad9253ffa02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:01.316323 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e9726-985c-4236-9686-ad9253ffa02f None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:01.316609 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e9726-985c-4236-9686-ad9253ffa02f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:01.316887 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e9726-985c-4236-9686-ad9253ffa02f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:01.318047 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248e9726-985c-4236-9686-ad9253ffa02f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:01.319522 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248e9726-985c-4236-9686-ad9253ffa02f None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:01.321131 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 26/52] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:24:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:01.327627 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab7361-4f3d-4230-8b7d-fba5af6ad6bd None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:01.417141 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:01.426508 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab7361-4f3d-4230-8b7d-fba5af6ad6bd None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:01.460324 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf01a2-0215-48d3-b2f0-cd9ca9b6b402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:01.460971 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf01a2-0215-48d3-b2f0-cd9ca9b6b402 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:01.461254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf01a2-0215-48d3-b2f0-cd9ca9b6b402 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:01.461531 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf01a2-0215-48d3-b2f0-cd9ca9b6b402 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:01.462399 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbf01a2-0215-48d3-b2f0-cd9ca9b6b402 None admin] RBAC: Adding query filter params (role.id=4f736313c26d4f8db48bbb2f346e50e4, scope.domain.id=default, user.id=d018cc34dbeb4d56a3052de5c20b961c) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:01.462711 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbf01a2-0215-48d3-b2f0-cd9ca9b6b402 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:01.464160 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbf01a2-0215-48d3-b2f0-cd9ca9b6b402 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:01.469950 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 27/53] 10.222.0.15 () {56 vars in 1322 bytes} [Wed Jun 15 19:24:01 2022] GET /identity/v3/role_assignments?user.id=d018cc34dbeb4d56a3052de5c20b961c&scope.domain.id=default&role.id=4f736313c26d4f8db48bbb2f346e50e4 => generated 530 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:01.564790 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd59c7-14f9-4934-a6d3-0aa76387c711 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:01.565075 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd59c7-14f9-4934-a6d3-0aa76387c711 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:01.565353 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd59c7-14f9-4934-a6d3-0aa76387c711 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:01.566787 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 27/54] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:01.572267 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59efc5e-4e63-4714-8d4c-3b5e2788460d None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:01.572560 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59efc5e-4e63-4714-8d4c-3b5e2788460d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:01.572841 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59efc5e-4e63-4714-8d4c-3b5e2788460d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:01.590128 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-b59efc5e-4e63-4714-8d4c-3b5e2788460d None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:01.591871 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b59efc5e-4e63-4714-8d4c-3b5e2788460d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:01.662430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 28/55] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:01.685690 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd59c7-14f9-4934-a6d3-0aa76387c711 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:01.784887 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd59c7-14f9-4934-a6d3-0aa76387c711 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:01.801219 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:01.817456 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c117f6-4df5-4eef-b0a8-41be6433f0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:01.818270 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c117f6-4df5-4eef-b0a8-41be6433f0f2 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:01.818553 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c117f6-4df5-4eef-b0a8-41be6433f0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:01.818826 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c117f6-4df5-4eef-b0a8-41be6433f0f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:01.819984 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c117f6-4df5-4eef-b0a8-41be6433f0f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:01.821423 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c117f6-4df5-4eef-b0a8-41be6433f0f2 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:01.823052 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 28/56] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:24:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:01.829204 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59efc5e-4e63-4714-8d4c-3b5e2788460d None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:01.910886 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:01.925667 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59efc5e-4e63-4714-8d4c-3b5e2788460d None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:01.958261 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afd36df-9bf9-474f-b870-edb3342cc9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:01.958850 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afd36df-9bf9-474f-b870-edb3342cc9a6 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:01.959135 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afd36df-9bf9-474f-b870-edb3342cc9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:01.959412 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afd36df-9bf9-474f-b870-edb3342cc9a6 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:01.960120 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afd36df-9bf9-474f-b870-edb3342cc9a6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:01.961560 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afd36df-9bf9-474f-b870-edb3342cc9a6 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:01.985404 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 29/57] 10.222.0.15 () {60 vars in 1140 bytes} [Wed Jun 15 19:24:01 2022] POST /identity/v3/projects => generated 298 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:24:04.073713 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726173a-e164-4aa5-b76d-8b89e4e254a8 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:04.074249 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726173a-e164-4aa5-b76d-8b89e4e254a8 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:04.074392 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726173a-e164-4aa5-b76d-8b89e4e254a8 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:04.075792 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 29/58] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:04.080932 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f477dc9-f164-419e-9149-de147d1b9573 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:04.081319 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f477dc9-f164-419e-9149-de147d1b9573 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:04.081498 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f477dc9-f164-419e-9149-de147d1b9573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:04.097642 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-2f477dc9-f164-419e-9149-de147d1b9573 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:04.098847 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-2f477dc9-f164-419e-9149-de147d1b9573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:04.162145 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 30/59] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:04.186736 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726173a-e164-4aa5-b76d-8b89e4e254a8 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:04.284845 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726173a-e164-4aa5-b76d-8b89e4e254a8 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:04.317304 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaedc19-bf49-404e-a5cd-277be7f10959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:04.318108 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaedc19-bf49-404e-a5cd-277be7f10959 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:04.318399 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaedc19-bf49-404e-a5cd-277be7f10959 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:04.318683 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaedc19-bf49-404e-a5cd-277be7f10959 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:04.319852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaedc19-bf49-404e-a5cd-277be7f10959 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:04.321295 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaedc19-bf49-404e-a5cd-277be7f10959 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:04.322961 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 30/60] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:24:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:04.328243 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f477dc9-f164-419e-9149-de147d1b9573 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:04.393666 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:04.427610 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f477dc9-f164-419e-9149-de147d1b9573 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:04.459878 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f6a60-7f1a-4001-b961-a45bc26338f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:04.460547 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f6a60-7f1a-4001-b961-a45bc26338f4 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:04.460833 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f6a60-7f1a-4001-b961-a45bc26338f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:04.461113 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f6a60-7f1a-4001-b961-a45bc26338f4 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:04.461836 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134f6a60-7f1a-4001-b961-a45bc26338f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:04.463314 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134f6a60-7f1a-4001-b961-a45bc26338f4 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:04.488161 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 31/61] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Jun 15 19:24:04 2022] POST /identity/v3/users => generated 275 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:24:07.761932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641628e-0e89-4938-97b1-05742e47d08a None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:07.763381 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641628e-0e89-4938-97b1-05742e47d08a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:07.763810 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641628e-0e89-4938-97b1-05742e47d08a None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:07.765341 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 31/62] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:07.770811 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a407b84-8dc2-44ea-b378-6bc4ca5487fa None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:07.771557 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a407b84-8dc2-44ea-b378-6bc4ca5487fa None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:07.772006 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a407b84-8dc2-44ea-b378-6bc4ca5487fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:07.793999 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-0a407b84-8dc2-44ea-b378-6bc4ca5487fa None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:07.795969 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0a407b84-8dc2-44ea-b378-6bc4ca5487fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:07.883761 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 32/63] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:07.923804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641628e-0e89-4938-97b1-05742e47d08a None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:08.003913 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b85d51-8b47-489a-b744-047c9aee688f None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:08.004447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b85d51-8b47-489a-b744-047c9aee688f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:08.004876 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b85d51-8b47-489a-b744-047c9aee688f None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:08.006553 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 33/64] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:08.010295 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:08.016392 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dca26d-7930-401e-b34a-ef6b965a14c0 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:08.017005 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dca26d-7930-401e-b34a-ef6b965a14c0 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:08.017485 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dca26d-7930-401e-b34a-ef6b965a14c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:08.036452 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-87dca26d-7930-401e-b34a-ef6b965a14c0 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:08.038144 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-87dca26d-7930-401e-b34a-ef6b965a14c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:08.087791 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641628e-0e89-4938-97b1-05742e47d08a None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:08.130900 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:08.132444 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d00cfe-1cf3-4e3b-baa0-cdf42916bb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:08.133469 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d00cfe-1cf3-4e3b-baa0-cdf42916bb94 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:08.134238 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d00cfe-1cf3-4e3b-baa0-cdf42916bb94 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:08.134780 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d00cfe-1cf3-4e3b-baa0-cdf42916bb94 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:08.150690 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d00cfe-1cf3-4e3b-baa0-cdf42916bb94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:08.151380 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 34/65] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 19:24:08.152818 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460e03e-e20b-4f7f-aca9-264d923df4a3 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:08.153278 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460e03e-e20b-4f7f-aca9-264d923df4a3 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:08.154259 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460e03e-e20b-4f7f-aca9-264d923df4a3 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:08.155730 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 35/66] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:08.157037 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8deaa2-4cfe-4ea6-93d9-70dada0b1b08 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:08.157503 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8deaa2-4cfe-4ea6-93d9-70dada0b1b08 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:08.157956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8deaa2-4cfe-4ea6-93d9-70dada0b1b08 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:08.159514 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 36/67] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:08.160837 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8403a4-0fc6-4ced-951c-e78fb4cf9d04 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:08.161339 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8403a4-0fc6-4ced-951c-e78fb4cf9d04 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:08.161831 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8403a4-0fc6-4ced-951c-e78fb4cf9d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:08.165409 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-17d00cfe-1cf3-4e3b-baa0-cdf42916bb94 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 19:24:08.165409 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:08.165409 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:08.165409 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:08.165409 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:08.165409 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:08.165409 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:08.165409 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:08.165409 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:08.165409 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:08.165409 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:08.165409 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:08.165409 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:08.165409 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:08.165409 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:08.165409 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:08.166395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:08.166395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:08.166395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:08.166395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:08.166395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:08.166395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:08.166395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:08.166395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:08.166395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:08.166395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:08.166395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:08.166395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:08.166395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:08.166395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:08.166395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:08.166395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:08.167179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:08.167179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:08.167179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:08.167179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:08.167179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:08.167179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:08.167179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:08.167179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:08.167179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:08.167179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:08.167179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 19:24:08.167179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:24:08.169907 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 32/68] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:24:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:08.171354 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9404da4-712d-420a-9a76-1d5a9656d2f4 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:08.171824 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9404da4-712d-420a-9a76-1d5a9656d2f4 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:08.172262 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9404da4-712d-420a-9a76-1d5a9656d2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:08.184685 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-2f8403a4-0fc6-4ced-951c-e78fb4cf9d04 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:08.186448 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-2f8403a4-0fc6-4ced-951c-e78fb4cf9d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:08.190652 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-c9404da4-712d-420a-9a76-1d5a9656d2f4 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:08.192032 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-c9404da4-712d-420a-9a76-1d5a9656d2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:08.259995 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 37/69] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:08.260866 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8403a4-0fc6-4ced-951c-e78fb4cf9d04 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:08.277384 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 33/70] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:08.278798 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9404da4-712d-420a-9a76-1d5a9656d2f4 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:08.334609 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:08.368639 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8403a4-0fc6-4ced-951c-e78fb4cf9d04 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:08.382786 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:08.389881 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9404da4-712d-420a-9a76-1d5a9656d2f4 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:08.403055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff72c3-b043-4b64-95e3-c167ffb35ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:08.403756 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff72c3-b043-4b64-95e3-c167ffb35ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:08.404043 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff72c3-b043-4b64-95e3-c167ffb35ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:08.404323 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff72c3-b043-4b64-95e3-c167ffb35ba6 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:08.405085 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff72c3-b043-4b64-95e3-c167ffb35ba6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:08.405395 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff72c3-b043-4b64-95e3-c167ffb35ba6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:08.406826 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff72c3-b043-4b64-95e3-c167ffb35ba6 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:08.414811 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 38/71] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jun 15 19:24:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:08.416115 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583afa7-cd00-4f83-9dc8-a0afbb40438d None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:08.416384 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583afa7-cd00-4f83-9dc8-a0afbb40438d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:08.416666 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583afa7-cd00-4f83-9dc8-a0afbb40438d None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:08.418050 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 39/72] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:08.418696 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c583afa7-cd00-4f83-9dc8-a0afbb40438d None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:08.427013 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17f44f-2385-472c-b305-2396a000fb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:08.427816 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17f44f-2385-472c-b305-2396a000fb8f None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:08.428101 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17f44f-2385-472c-b305-2396a000fb8f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:08.428382 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17f44f-2385-472c-b305-2396a000fb8f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:08.440959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca17f44f-2385-472c-b305-2396a000fb8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:08.448598 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-ca17f44f-2385-472c-b305-2396a000fb8f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:24:08.448598 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:08.448598 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:08.448598 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:08.448598 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:08.448598 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:08.448598 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:08.448598 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:08.448598 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:08.448598 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:08.448598 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:08.448598 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:08.448598 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:08.448598 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:08.448598 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:08.448598 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:08.449454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:08.449454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:08.449454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:08.449454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:08.449454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:08.449454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:08.449454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:08.449454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:08.449454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:08.449454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:08.449454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:08.449454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:08.449454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:08.449454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:08.449454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:08.449454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:08.450229 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:08.450229 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:08.450229 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:08.450229 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:08.450229 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:08.450229 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:08.450229 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:08.450229 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:08.450229 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:08.450229 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:08.450229 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:24:08.450229 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:24:08.450229 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 34/73] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jun 15 19:24:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:08.450889 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17f44f-2385-472c-b305-2396a000fb8f None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:08.520908 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c583afa7-cd00-4f83-9dc8-a0afbb40438d None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:08.547152 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17f44f-2385-472c-b305-2396a000fb8f None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:08.549288 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:08.554249 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1870214e-8e8d-44bd-975f-dbf0c8f74f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:08.555061 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870214e-8e8d-44bd-975f-dbf0c8f74f60 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:08.555348 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870214e-8e8d-44bd-975f-dbf0c8f74f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:08.555625 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870214e-8e8d-44bd-975f-dbf0c8f74f60 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:08.565945 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1870214e-8e8d-44bd-975f-dbf0c8f74f60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:08.574055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-1870214e-8e8d-44bd-975f-dbf0c8f74f60 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 19:24:08.574055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:08.574055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:08.574055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:08.574055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:08.574055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:08.574055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:08.574055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:08.574055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:08.574055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:08.574055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:08.574055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:08.574055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:08.574055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:08.574055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:08.574055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:08.574920 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:08.574920 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:08.574920 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:08.574920 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:08.574920 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:08.574920 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:08.574920 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:08.574920 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:08.574920 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:08.574920 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:08.574920 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:08.574920 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:08.574920 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:08.574920 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:08.574920 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:08.574920 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:08.576381 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:08.576381 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:08.576381 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:08.576381 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:08.576381 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:08.576381 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:08.576381 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:08.576381 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:08.576381 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:08.576381 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:08.576381 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 19:24:08.576381 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:24:08.576381 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 40/74] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jun 15 19:24:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:08.577484 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1870214e-8e8d-44bd-975f-dbf0c8f74f60 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:08.584607 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7ce58-df9a-4411-915b-f7ca15e0001e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:08.585404 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7ce58-df9a-4411-915b-f7ca15e0001e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:08.585690 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7ce58-df9a-4411-915b-f7ca15e0001e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:08.585970 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7ce58-df9a-4411-915b-f7ca15e0001e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:08.596201 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e7ce58-df9a-4411-915b-f7ca15e0001e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:08.603868 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-22e7ce58-df9a-4411-915b-f7ca15e0001e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 19:24:08.603868 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:08.603868 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:08.603868 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:08.603868 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:08.603868 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:08.603868 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:08.603868 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:08.603868 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:08.603868 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:08.603868 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:08.603868 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:08.603868 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:08.603868 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:08.603868 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:08.603868 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:08.604704 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:08.604704 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:08.604704 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:08.604704 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:08.604704 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:08.604704 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:08.604704 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:08.604704 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:08.604704 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:08.604704 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:08.604704 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:08.604704 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:08.604704 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:08.604704 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:08.604704 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:08.604704 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:08.610830 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:08.610830 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:08.610830 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:08.610830 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:08.610830 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:08.610830 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:08.610830 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:08.610830 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:08.610830 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:08.610830 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:08.610830 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 19:24:08.610830 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:24:08.610830 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 35/75] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:24:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:08.610830 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ad761-bf65-4f42-8e7f-696a4726e2a1 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:08.611500 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ad761-bf65-4f42-8e7f-696a4726e2a1 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:08.611500 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ad761-bf65-4f42-8e7f-696a4726e2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:08.611611 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:08.623615 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-519ad761-bf65-4f42-8e7f-696a4726e2a1 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:08.624813 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-519ad761-bf65-4f42-8e7f-696a4726e2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:08.675171 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1870214e-8e8d-44bd-975f-dbf0c8f74f60 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:08.689660 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 36/76] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:08.690507 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ad761-bf65-4f42-8e7f-696a4726e2a1 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:08.708853 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c621b9-6d43-4408-9655-43ff09a96e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:08.709728 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c621b9-6d43-4408-9655-43ff09a96e83 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:08.710017 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c621b9-6d43-4408-9655-43ff09a96e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:08.710320 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c621b9-6d43-4408-9655-43ff09a96e83 None admin] PATH_INFO: `/v3/users/abb9d50717324682b8fc92f386605834` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:08.724207 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c621b9-6d43-4408-9655-43ff09a96e83 None admin] RBAC: Authorizing `identity:get_user(user_id=abb9d50717324682b8fc92f386605834)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:08.725656 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c621b9-6d43-4408-9655-43ff09a96e83 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:08.727359 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 41/77] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:24:08 2022] GET /identity/v3/users/abb9d50717324682b8fc92f386605834 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:08.728118 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c621b9-6d43-4408-9655-43ff09a96e83 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:08.791467 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:08.798480 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ad761-bf65-4f42-8e7f-696a4726e2a1 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:08.818684 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:08.826226 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c621b9-6d43-4408-9655-43ff09a96e83 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:08.832351 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c01b37-4e55-410c-9d48-385f880e028b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:08.833017 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c01b37-4e55-410c-9d48-385f880e028b None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:08.833306 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c01b37-4e55-410c-9d48-385f880e028b None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:08.833584 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c01b37-4e55-410c-9d48-385f880e028b None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:08.834368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c01b37-4e55-410c-9d48-385f880e028b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:08.834676 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c01b37-4e55-410c-9d48-385f880e028b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:08.836061 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c01b37-4e55-410c-9d48-385f880e028b None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:08.844113 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 37/78] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:24:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:08.844113 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c01b37-4e55-410c-9d48-385f880e028b None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:08.859118 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc14e69-2747-4209-99fa-64f4b65e0342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:08.859788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc14e69-2747-4209-99fa-64f4b65e0342 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:08.860071 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc14e69-2747-4209-99fa-64f4b65e0342 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:08.860350 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc14e69-2747-4209-99fa-64f4b65e0342 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:08.861090 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc14e69-2747-4209-99fa-64f4b65e0342 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:08.861398 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc14e69-2747-4209-99fa-64f4b65e0342 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:08.862802 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc14e69-2747-4209-99fa-64f4b65e0342 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:08.870501 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 42/79] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:24:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:08.871266 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc14e69-2747-4209-99fa-64f4b65e0342 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:08.945368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c01b37-4e55-410c-9d48-385f880e028b None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:08.971209 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc14e69-2747-4209-99fa-64f4b65e0342 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:08.978217 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4179ed-be64-41d0-bb62-4082b34456e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:08.978876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4179ed-be64-41d0-bb62-4082b34456e8 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:08.979161 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4179ed-be64-41d0-bb62-4082b34456e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:08.979439 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4179ed-be64-41d0-bb62-4082b34456e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:08.980186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4179ed-be64-41d0-bb62-4082b34456e8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:08.980490 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4179ed-be64-41d0-bb62-4082b34456e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:08.981902 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4179ed-be64-41d0-bb62-4082b34456e8 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:08.989768 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 38/80] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jun 15 19:24:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:08.990591 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4179ed-be64-41d0-bb62-4082b34456e8 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:09.005331 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8da4ad-e888-48f0-9540-91d852581b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:09.006146 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8da4ad-e888-48f0-9540-91d852581b2d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:09.006436 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8da4ad-e888-48f0-9540-91d852581b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:09.006712 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8da4ad-e888-48f0-9540-91d852581b2d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:09.007870 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8da4ad-e888-48f0-9540-91d852581b2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:09.009572 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8da4ad-e888-48f0-9540-91d852581b2d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:09.011370 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 43/81] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:24:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:09.011476 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:09.011953 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8da4ad-e888-48f0-9540-91d852581b2d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:09.037117 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:09.096759 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4179ed-be64-41d0-bb62-4082b34456e8 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:09.112246 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8da4ad-e888-48f0-9540-91d852581b2d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:09.132156 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5947cdb5-66be-4e4e-b2d7-7b491620bb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:09.132807 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947cdb5-66be-4e4e-b2d7-7b491620bb67 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:09.133092 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947cdb5-66be-4e4e-b2d7-7b491620bb67 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:09.133377 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947cdb5-66be-4e4e-b2d7-7b491620bb67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:09.147821 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da7cb95-17e0-4a8d-8064-9cf201bb5cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:09.148694 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7cb95-17e0-4a8d-8064-9cf201bb5cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:09.148979 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7cb95-17e0-4a8d-8064-9cf201bb5cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:09.149254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7cb95-17e0-4a8d-8064-9cf201bb5cf6 None admin] PATH_INFO: `/v3/users/d018cc34dbeb4d56a3052de5c20b961c` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:09.150493 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da7cb95-17e0-4a8d-8064-9cf201bb5cf6 None admin] RBAC: Authorizing `identity:get_user(user_id=d018cc34dbeb4d56a3052de5c20b961c)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:09.151939 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da7cb95-17e0-4a8d-8064-9cf201bb5cf6 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:09.153734 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 44/82] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:24:09 2022] GET /identity/v3/users/d018cc34dbeb4d56a3052de5c20b961c => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:09.154354 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da7cb95-17e0-4a8d-8064-9cf201bb5cf6 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:09.204063 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5947cdb5-66be-4e4e-b2d7-7b491620bb67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:09.205945 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5947cdb5-66be-4e4e-b2d7-7b491620bb67 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:09.223086 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:09.241725 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:09.251377 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da7cb95-17e0-4a8d-8064-9cf201bb5cf6 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:09.283604 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58709220-674e-4257-ba1c-5573d461f118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:09.284471 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58709220-674e-4257-ba1c-5573d461f118 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:09.284759 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58709220-674e-4257-ba1c-5573d461f118 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:09.285040 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58709220-674e-4257-ba1c-5573d461f118 None admin] PATH_INFO: `/v3/users/abb9d50717324682b8fc92f386605834` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:09.286268 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58709220-674e-4257-ba1c-5573d461f118 None admin] RBAC: Authorizing `identity:get_user(user_id=abb9d50717324682b8fc92f386605834)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:09.287697 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58709220-674e-4257-ba1c-5573d461f118 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:09.289324 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 45/83] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:24:09 2022] GET /identity/v3/users/abb9d50717324682b8fc92f386605834 => generated 275 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:09.293565 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58709220-674e-4257-ba1c-5573d461f118 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:09.305436 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 39/84] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:24:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 19:24:09.306278 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5947cdb5-66be-4e4e-b2d7-7b491620bb67 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:09.399432 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58709220-674e-4257-ba1c-5573d461f118 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:09.403951 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5947cdb5-66be-4e4e-b2d7-7b491620bb67 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:09.420983 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:09.432429 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a763a-1a68-411c-b237-b647c77644cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:09.433305 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a763a-1a68-411c-b237-b647c77644cc None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:09.433594 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a763a-1a68-411c-b237-b647c77644cc None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:09.433872 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a763a-1a68-411c-b237-b647c77644cc None admin] PATH_INFO: `/v3/users/abb9d50717324682b8fc92f386605834` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:09.435225 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72a763a-1a68-411c-b237-b647c77644cc None admin] RBAC: Authorizing `identity:get_user(user_id=abb9d50717324682b8fc92f386605834)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:09.436601 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72a763a-1a68-411c-b237-b647c77644cc None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:09.437945 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8f5ab0-d4d2-4b8a-b1ea-b1cebbc0865f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:09.438255 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 46/85] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:24:09 2022] GET /identity/v3/users/abb9d50717324682b8fc92f386605834 => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:09.438644 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f5ab0-d4d2-4b8a-b1ea-b1cebbc0865f None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:09.438927 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f5ab0-d4d2-4b8a-b1ea-b1cebbc0865f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:09.439016 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a763a-1a68-411c-b237-b647c77644cc None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:09.439249 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f5ab0-d4d2-4b8a-b1ea-b1cebbc0865f None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:09.439912 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8f5ab0-d4d2-4b8a-b1ea-b1cebbc0865f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:09.441340 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8f5ab0-d4d2-4b8a-b1ea-b1cebbc0865f None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:09.467070 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 40/86] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Jun 15 19:24:09 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:24:09.467904 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8f5ab0-d4d2-4b8a-b1ea-b1cebbc0865f None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:09.485619 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:09.568751 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a763a-1a68-411c-b237-b647c77644cc None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:09.585768 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8f5ab0-d4d2-4b8a-b1ea-b1cebbc0865f None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:09.609912 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f268650-76a9-4dc2-85f5-2d69c0233c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:09.611130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f268650-76a9-4dc2-85f5-2d69c0233c3a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:09.611997 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f268650-76a9-4dc2-85f5-2d69c0233c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:09.612456 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f268650-76a9-4dc2-85f5-2d69c0233c3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:09.627528 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f5916-3cb6-42cc-a802-f7430c3bb842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:09.628055 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f5916-3cb6-42cc-a802-f7430c3bb842 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:09.628440 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f5916-3cb6-42cc-a802-f7430c3bb842 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:09.628814 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f5916-3cb6-42cc-a802-f7430c3bb842 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:09.676354 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:09.707083 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f268650-76a9-4dc2-85f5-2d69c0233c3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:09.708014 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f5916-3cb6-42cc-a802-f7430c3bb842 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:09.709061 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f268650-76a9-4dc2-85f5-2d69c0233c3a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:09.709499 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f5916-3cb6-42cc-a802-f7430c3bb842 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:09.747799 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:09.831619 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 47/87] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:24:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 19:24:09.832477 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f268650-76a9-4dc2-85f5-2d69c0233c3a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:09.836161 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 41/88] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:24:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 19:24:09.837092 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f5916-3cb6-42cc-a802-f7430c3bb842 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:09.916666 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:09.944080 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f5916-3cb6-42cc-a802-f7430c3bb842 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:09.974416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f268650-76a9-4dc2-85f5-2d69c0233c3a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:09.986376 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2347283-c001-4908-8482-4e2ea5dd8838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:09.987543 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2347283-c001-4908-8482-4e2ea5dd8838 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:09.987840 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2347283-c001-4908-8482-4e2ea5dd8838 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:09.988128 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2347283-c001-4908-8482-4e2ea5dd8838 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:10.005682 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:10.017295 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc09de-122e-44e3-9e64-49c456dbf042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:10.018201 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc09de-122e-44e3-9e64-49c456dbf042 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:10.018486 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc09de-122e-44e3-9e64-49c456dbf042 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:10.018762 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc09de-122e-44e3-9e64-49c456dbf042 None admin] PATH_INFO: `/v3/projects/c4f989e5fad14bcebae5e279f22ee21d` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:10.019983 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc09de-122e-44e3-9e64-49c456dbf042 None admin] RBAC: Authorizing `identity:get_project(project_id=c4f989e5fad14bcebae5e279f22ee21d)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:10.021439 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc09de-122e-44e3-9e64-49c456dbf042 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:10.023242 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 48/89] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:09 2022] GET /identity/v3/projects/c4f989e5fad14bcebae5e279f22ee21d => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:10.024011 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc09de-122e-44e3-9e64-49c456dbf042 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:10.087808 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2347283-c001-4908-8482-4e2ea5dd8838 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:10.089527 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2347283-c001-4908-8482-4e2ea5dd8838 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:10.139903 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc09de-122e-44e3-9e64-49c456dbf042 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:10.173780 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:10.176161 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19e42bd-a914-4a76-a5d7-6f6463e7eac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:10.177255 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e42bd-a914-4a76-a5d7-6f6463e7eac1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:10.177725 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e42bd-a914-4a76-a5d7-6f6463e7eac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:10.178251 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e42bd-a914-4a76-a5d7-6f6463e7eac1 None admin] PATH_INFO: `/v3/projects/c4f989e5fad14bcebae5e279f22ee21d` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:10.179733 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19e42bd-a914-4a76-a5d7-6f6463e7eac1 None admin] RBAC: Authorizing `identity:get_project(project_id=c4f989e5fad14bcebae5e279f22ee21d)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:10.181384 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19e42bd-a914-4a76-a5d7-6f6463e7eac1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:10.183348 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 49/90] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:10 2022] GET /identity/v3/projects/c4f989e5fad14bcebae5e279f22ee21d => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:10.184338 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19e42bd-a914-4a76-a5d7-6f6463e7eac1 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:10.202939 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 42/91] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:24:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 19:24:10.206139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2347283-c001-4908-8482-4e2ea5dd8838 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:10.262537 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:10.297514 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19e42bd-a914-4a76-a5d7-6f6463e7eac1 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:10.306941 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2347283-c001-4908-8482-4e2ea5dd8838 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:10.335322 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4f266-136f-4a0e-9b40-154cf76cb159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:10.336302 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4f266-136f-4a0e-9b40-154cf76cb159 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:10.336662 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4f266-136f-4a0e-9b40-154cf76cb159 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:10.337088 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4f266-136f-4a0e-9b40-154cf76cb159 None admin] PATH_INFO: `/v3/projects/c4f989e5fad14bcebae5e279f22ee21d` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:10.338492 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa4f266-136f-4a0e-9b40-154cf76cb159 None admin] RBAC: Authorizing `identity:get_project(project_id=c4f989e5fad14bcebae5e279f22ee21d)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:10.340126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa4f266-136f-4a0e-9b40-154cf76cb159 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:10.342595 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 50/92] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:10 2022] GET /identity/v3/projects/c4f989e5fad14bcebae5e279f22ee21d => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:10.346830 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4f266-136f-4a0e-9b40-154cf76cb159 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:10.353211 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a7c44-1f14-48cd-afeb-7a7a47ba1a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:10.354187 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a7c44-1f14-48cd-afeb-7a7a47ba1a2b None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:10.354719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a7c44-1f14-48cd-afeb-7a7a47ba1a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:10.355193 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a7c44-1f14-48cd-afeb-7a7a47ba1a2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:10.356274 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a7c44-1f14-48cd-afeb-7a7a47ba1a2b None admin] RBAC: Adding query filter params (role.id=e53a78912e764f8387e0cf637bb86bee, scope.project.id=c4f989e5fad14bcebae5e279f22ee21d, user.id=abb9d50717324682b8fc92f386605834) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:10.357806 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a7c44-1f14-48cd-afeb-7a7a47ba1a2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:10.360049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a7c44-1f14-48cd-afeb-7a7a47ba1a2b None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:10.372778 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 43/93] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:24:10 2022] GET /identity/v3/role_assignments?user.id=abb9d50717324682b8fc92f386605834&scope.project.id=c4f989e5fad14bcebae5e279f22ee21d&role.id=e53a78912e764f8387e0cf637bb86bee => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:10.374220 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a7c44-1f14-48cd-afeb-7a7a47ba1a2b None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:10.459345 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:10.561014 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4f266-136f-4a0e-9b40-154cf76cb159 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:10.569993 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a7c44-1f14-48cd-afeb-7a7a47ba1a2b None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:10.594025 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:10.599970 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3410a78-6a2f-4459-a4bf-6993f8990d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:10.600877 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3410a78-6a2f-4459-a4bf-6993f8990d3e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:10.601416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3410a78-6a2f-4459-a4bf-6993f8990d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:10.601889 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3410a78-6a2f-4459-a4bf-6993f8990d3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:10.603811 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3410a78-6a2f-4459-a4bf-6993f8990d3e None admin] RBAC: Adding query filter params (role.id=4f736313c26d4f8db48bbb2f346e50e4, scope.project.id=c4f989e5fad14bcebae5e279f22ee21d, user.id=d018cc34dbeb4d56a3052de5c20b961c) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:10.604363 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3410a78-6a2f-4459-a4bf-6993f8990d3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:10.606189 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3410a78-6a2f-4459-a4bf-6993f8990d3e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:10.611370 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd273ab8-739c-4a84-b402-f5b189c30048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:10.612241 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd273ab8-739c-4a84-b402-f5b189c30048 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:10.612531 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd273ab8-739c-4a84-b402-f5b189c30048 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:10.612818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd273ab8-739c-4a84-b402-f5b189c30048 None admin] PATH_INFO: `/v3/projects/0c11dde50af14022b1a1fe4e7a2f8a94` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:10.613882 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 51/94] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:24:10 2022] GET /identity/v3/role_assignments?user.id=d018cc34dbeb4d56a3052de5c20b961c&scope.project.id=c4f989e5fad14bcebae5e279f22ee21d&role.id=4f736313c26d4f8db48bbb2f346e50e4 => generated 259 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:10.614053 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd273ab8-739c-4a84-b402-f5b189c30048 None admin] RBAC: Authorizing `identity:get_project(project_id=0c11dde50af14022b1a1fe4e7a2f8a94)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:10.615116 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3410a78-6a2f-4459-a4bf-6993f8990d3e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:10.615625 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd273ab8-739c-4a84-b402-f5b189c30048 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:10.617388 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 44/95] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:10 2022] GET /identity/v3/projects/0c11dde50af14022b1a1fe4e7a2f8a94 => generated 312 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:10.639573 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd273ab8-739c-4a84-b402-f5b189c30048 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:10.733757 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3410a78-6a2f-4459-a4bf-6993f8990d3e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:10.771578 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae291db6-5599-4f20-aecd-e0310a416952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:10.772694 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae291db6-5599-4f20-aecd-e0310a416952 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:10.773207 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae291db6-5599-4f20-aecd-e0310a416952 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:10.773765 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae291db6-5599-4f20-aecd-e0310a416952 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:10.774896 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae291db6-5599-4f20-aecd-e0310a416952 None admin] RBAC: Adding query filter params (role.id=3920d5126e22480ab9a4eec9cf0fd4e7, scope.project.id=c4f989e5fad14bcebae5e279f22ee21d, user.id=abb9d50717324682b8fc92f386605834) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:10.775462 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae291db6-5599-4f20-aecd-e0310a416952 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:10.777180 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae291db6-5599-4f20-aecd-e0310a416952 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:10.783909 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 52/96] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:24:10 2022] GET /identity/v3/role_assignments?user.id=abb9d50717324682b8fc92f386605834&scope.project.id=c4f989e5fad14bcebae5e279f22ee21d&role.id=3920d5126e22480ab9a4eec9cf0fd4e7 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:10.863477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd273ab8-739c-4a84-b402-f5b189c30048 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:10.875227 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:10.919641 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2709b0-55d4-41e3-b921-a50cb21f2585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:10.920348 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2709b0-55d4-41e3-b921-a50cb21f2585 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:10.920641 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2709b0-55d4-41e3-b921-a50cb21f2585 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:10.920922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2709b0-55d4-41e3-b921-a50cb21f2585 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:10.921790 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2709b0-55d4-41e3-b921-a50cb21f2585 None admin] RBAC: Adding query filter params (role.id=e53a78912e764f8387e0cf637bb86bee, scope.project.id=0c11dde50af14022b1a1fe4e7a2f8a94, user.id=abb9d50717324682b8fc92f386605834) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:10.922289 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2709b0-55d4-41e3-b921-a50cb21f2585 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:10.923813 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2709b0-55d4-41e3-b921-a50cb21f2585 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:10.930769 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 45/97] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:24:10 2022] GET /identity/v3/role_assignments?user.id=abb9d50717324682b8fc92f386605834&scope.project.id=0c11dde50af14022b1a1fe4e7a2f8a94&role.id=e53a78912e764f8387e0cf637bb86bee => generated 259 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:12.139947 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12a885-7dcb-42a0-97dc-be591d7c2f60 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:12.140453 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12a885-7dcb-42a0-97dc-be591d7c2f60 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:12.140526 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12a885-7dcb-42a0-97dc-be591d7c2f60 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:12.142035 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 53/98] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:12.147442 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9ca07-7814-4692-910e-e7f116482874 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:12.147853 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9ca07-7814-4692-910e-e7f116482874 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:12.148018 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9ca07-7814-4692-910e-e7f116482874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:12.170719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-7ee9ca07-7814-4692-910e-e7f116482874 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:12.171929 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-7ee9ca07-7814-4692-910e-e7f116482874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:12.264793 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 46/99] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:12.307790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12a885-7dcb-42a0-97dc-be591d7c2f60 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:12.402333 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f8fbe-ea6f-4391-9796-26090a9526bc None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:12.402830 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f8fbe-ea6f-4391-9796-26090a9526bc None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:12.403321 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f8fbe-ea6f-4391-9796-26090a9526bc None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:12.404985 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 47/100] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:12.411021 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:12.415237 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab411b4-6447-4ea2-b971-11dfefbdfb6a None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:12.415829 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab411b4-6447-4ea2-b971-11dfefbdfb6a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:12.416307 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab411b4-6447-4ea2-b971-11dfefbdfb6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:12.458910 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-5ab411b4-6447-4ea2-b971-11dfefbdfb6a None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:12.461149 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-5ab411b4-6447-4ea2-b971-11dfefbdfb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:12.518187 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12a885-7dcb-42a0-97dc-be591d7c2f60 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:12.592224 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e89fe9-7f9d-4ba5-b0e4-a01419eb5a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:12.593384 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e89fe9-7f9d-4ba5-b0e4-a01419eb5a54 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:12.593888 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e89fe9-7f9d-4ba5-b0e4-a01419eb5a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:12.594378 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e89fe9-7f9d-4ba5-b0e4-a01419eb5a54 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:12.623636 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e89fe9-7f9d-4ba5-b0e4-a01419eb5a54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:12.624516 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 48/101] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 19:24:12.632029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-e1e89fe9-7f9d-4ba5-b0e4-a01419eb5a54 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 19:24:12.632029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:12.632029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:12.632029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:12.632029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:12.632029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:12.632029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:12.632029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:12.632029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:12.632029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:12.632029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:12.632029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:12.632029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:12.632029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:12.632029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:12.632029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:12.632865 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:12.632865 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:12.632865 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:12.632865 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:12.632865 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:12.632865 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:12.632865 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:12.632865 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:12.632865 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:12.632865 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:12.632865 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:12.632865 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:12.632865 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:12.632865 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:12.632865 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:12.632865 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:12.633678 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:12.633678 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:12.633678 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:12.633678 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:12.633678 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:12.633678 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:12.633678 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:12.633678 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:12.633678 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:12.633678 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:12.633678 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 19:24:12.633678 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:24:12.640832 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 54/102] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:24:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:12.646782 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab411b4-6447-4ea2-b971-11dfefbdfb6a None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:12.671629 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e89fe9-7f9d-4ba5-b0e4-a01419eb5a54 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:12.697230 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:12.843967 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab411b4-6447-4ea2-b971-11dfefbdfb6a None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:12.901843 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e89fe9-7f9d-4ba5-b0e4-a01419eb5a54 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:12.930952 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df4e7b-4842-4497-ba33-a304e3186646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:12.931927 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df4e7b-4842-4497-ba33-a304e3186646 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:12.932389 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df4e7b-4842-4497-ba33-a304e3186646 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:12.932840 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df4e7b-4842-4497-ba33-a304e3186646 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:12.933788 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3df4e7b-4842-4497-ba33-a304e3186646 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:12.934585 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3df4e7b-4842-4497-ba33-a304e3186646 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:12.936164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3df4e7b-4842-4497-ba33-a304e3186646 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:12.939203 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:12.947848 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 49/103] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jun 15 19:24:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:12.954012 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df4e7b-4842-4497-ba33-a304e3186646 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:12.975731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef762069-e57a-42df-9258-a58ea101de79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:12.976636 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef762069-e57a-42df-9258-a58ea101de79 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:12.976993 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef762069-e57a-42df-9258-a58ea101de79 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:12.977338 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef762069-e57a-42df-9258-a58ea101de79 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:12.978649 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef762069-e57a-42df-9258-a58ea101de79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:12.980276 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef762069-e57a-42df-9258-a58ea101de79 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:12.981933 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 55/104] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:24:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:24:12.987355 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef762069-e57a-42df-9258-a58ea101de79 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:13.154464 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:13.166355 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df4e7b-4842-4497-ba33-a304e3186646 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:13.178435 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef762069-e57a-42df-9258-a58ea101de79 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:13.238982 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9e653-083a-40c2-906b-6e68aa987d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:13.239806 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9e653-083a-40c2-906b-6e68aa987d7b None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:13.240099 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9e653-083a-40c2-906b-6e68aa987d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:13.240382 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9e653-083a-40c2-906b-6e68aa987d7b None admin] PATH_INFO: `/v3/users/a1ad396a5e26438288c0d36a4fe00a05` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:13.242047 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b27ea6-e84e-4c3e-bd1b-62d5d5f5cc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:13.242753 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b27ea6-e84e-4c3e-bd1b-62d5d5f5cc26 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:13.243041 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b27ea6-e84e-4c3e-bd1b-62d5d5f5cc26 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:13.243322 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b27ea6-e84e-4c3e-bd1b-62d5d5f5cc26 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:13.244048 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b27ea6-e84e-4c3e-bd1b-62d5d5f5cc26 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:13.246409 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b27ea6-e84e-4c3e-bd1b-62d5d5f5cc26 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:13.278925 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f9e653-083a-40c2-906b-6e68aa987d7b None admin] RBAC: Authorizing `identity:get_user(user_id=a1ad396a5e26438288c0d36a4fe00a05)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:13.280409 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f9e653-083a-40c2-906b-6e68aa987d7b None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:13.282195 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 50/105] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:24:12 2022] GET /identity/v3/users/a1ad396a5e26438288c0d36a4fe00a05 => generated 289 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:24:13.288034 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9e653-083a-40c2-906b-6e68aa987d7b None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:13.302174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 56/106] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Jun 15 19:24:12 2022] POST /identity/v3/projects => generated 302 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:24:13.422772 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9e653-083a-40c2-906b-6e68aa987d7b None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:13.465879 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c011e-541b-4cd1-8477-c1a542dbbc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:13.466586 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c011e-541b-4cd1-8477-c1a542dbbc6d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:13.466876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c011e-541b-4cd1-8477-c1a542dbbc6d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:13.467159 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c011e-541b-4cd1-8477-c1a542dbbc6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:13.514775 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:13.556812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c011e-541b-4cd1-8477-c1a542dbbc6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:13.558408 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c011e-541b-4cd1-8477-c1a542dbbc6d None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:13.667333 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 51/107] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:24:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 19:24:13.673519 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b27ea6-e84e-4c3e-bd1b-62d5d5f5cc26 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:13.742544 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:13.772698 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24a633-0360-4df8-b42e-b1877c8a5796 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:13.772983 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24a633-0360-4df8-b42e-b1877c8a5796 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:13.773194 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24a633-0360-4df8-b42e-b1877c8a5796 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:13.775718 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 52/108] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:13.786112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c431378-94b4-4178-b18a-665e4e96488a None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:13.786112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c431378-94b4-4178-b18a-665e4e96488a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:13.786112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c431378-94b4-4178-b18a-665e4e96488a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:13.802944 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-4c431378-94b4-4178-b18a-665e4e96488a None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:13.804171 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-4c431378-94b4-4178-b18a-665e4e96488a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:13.818833 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b27ea6-e84e-4c3e-bd1b-62d5d5f5cc26 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:13.866617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ff1d3a-1dcc-4f3c-b0a3-26bd31427c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:13.867992 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ff1d3a-1dcc-4f3c-b0a3-26bd31427c16 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:13.868287 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ff1d3a-1dcc-4f3c-b0a3-26bd31427c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:13.868570 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ff1d3a-1dcc-4f3c-b0a3-26bd31427c16 None admin] PATH_INFO: `/v3/projects/c4f989e5fad14bcebae5e279f22ee21d` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:13.869839 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ff1d3a-1dcc-4f3c-b0a3-26bd31427c16 None admin] RBAC: Authorizing `identity:get_project(project_id=c4f989e5fad14bcebae5e279f22ee21d)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:13.871649 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ff1d3a-1dcc-4f3c-b0a3-26bd31427c16 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:13.873427 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 57/109] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:13 2022] GET /identity/v3/projects/c4f989e5fad14bcebae5e279f22ee21d => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:13.879570 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ff1d3a-1dcc-4f3c-b0a3-26bd31427c16 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:13.922879 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:13.945415 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 53/110] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:13.955685 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b544534-b5af-4df0-ad71-69e2fee70a3a None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:13.956869 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b544534-b5af-4df0-ad71-69e2fee70a3a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:13.957511 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b544534-b5af-4df0-ad71-69e2fee70a3a None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:13.961534 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 54/111] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:13 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:13.963006 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f1e66-683c-4003-9337-c89a431b74d2 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:13.964296 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f1e66-683c-4003-9337-c89a431b74d2 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:13.964932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f1e66-683c-4003-9337-c89a431b74d2 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:13.968027 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 55/112] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:13.969016 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df6849-83db-4a4c-b43d-defba4086d37 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:13.969300 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df6849-83db-4a4c-b43d-defba4086d37 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:13.969582 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df6849-83db-4a4c-b43d-defba4086d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:13.989215 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-c0df6849-83db-4a4c-b43d-defba4086d37 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:13.990445 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-c0df6849-83db-4a4c-b43d-defba4086d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:14.022988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ff1d3a-1dcc-4f3c-b0a3-26bd31427c16 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:14.033359 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:14.061378 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf31b4b-eb06-4db9-bf0a-1da26476bdb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:14.062162 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf31b4b-eb06-4db9-bf0a-1da26476bdb0 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:14.062521 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 56/113] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:14.062751 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf31b4b-eb06-4db9-bf0a-1da26476bdb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:14.063017 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf31b4b-eb06-4db9-bf0a-1da26476bdb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:14.063806 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5df602-4a17-4da0-ae95-2df7d3ed84a6 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:14.063909 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf31b4b-eb06-4db9-bf0a-1da26476bdb0 None admin] RBAC: Adding query filter params (role.id=4e60cc1f881a4caebdc208088b7ffcaa, scope.project.id=c4f989e5fad14bcebae5e279f22ee21d, user.id=a1ad396a5e26438288c0d36a4fe00a05) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:14.064096 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5df602-4a17-4da0-ae95-2df7d3ed84a6 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:14.064192 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf31b4b-eb06-4db9-bf0a-1da26476bdb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:14.064375 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5df602-4a17-4da0-ae95-2df7d3ed84a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:14.065666 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf31b4b-eb06-4db9-bf0a-1da26476bdb0 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:14.072016 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 58/114] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:24:13 2022] GET /identity/v3/role_assignments?user.id=a1ad396a5e26438288c0d36a4fe00a05&scope.project.id=c4f989e5fad14bcebae5e279f22ee21d&role.id=4e60cc1f881a4caebdc208088b7ffcaa => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:14.072850 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf31b4b-eb06-4db9-bf0a-1da26476bdb0 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:14.103452 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-8e5df602-4a17-4da0-ae95-2df7d3ed84a6 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:14.104835 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-8e5df602-4a17-4da0-ae95-2df7d3ed84a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:14.192594 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf31b4b-eb06-4db9-bf0a-1da26476bdb0 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:14.193496 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 57/115] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:14.195365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5df602-4a17-4da0-ae95-2df7d3ed84a6 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:14.233366 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49eb07d-6192-4128-b294-2497190c3dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:14.234221 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49eb07d-6192-4128-b294-2497190c3dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:14.234511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49eb07d-6192-4128-b294-2497190c3dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:14.234791 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49eb07d-6192-4128-b294-2497190c3dc7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:14.247559 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49eb07d-6192-4128-b294-2497190c3dc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:14.255578 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:14.255793 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-c49eb07d-6192-4128-b294-2497190c3dc7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 19:24:14.255793 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:14.255793 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:14.255793 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:14.255793 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:14.255793 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:14.255793 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:14.255793 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:14.255793 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:14.255793 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:14.255793 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:14.255793 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:14.255793 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:14.255793 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:14.255793 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:14.255793 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:14.256573 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:14.256573 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:14.256573 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:14.256573 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:14.256573 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:14.256573 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:14.256573 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:14.256573 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:14.256573 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:14.256573 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:14.256573 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:14.256573 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:14.256573 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:14.256573 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:14.256573 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:14.256573 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:14.257911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:14.257911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:14.257911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:14.257911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:14.257911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:14.257911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:14.257911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:14.257911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:14.257911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:14.257911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:14.257911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 19:24:14.257911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:24:14.257911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 59/116] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:24:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:14.258772 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d57aa9-71c3-43a1-81b4-30088f69768d None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:14.258772 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d57aa9-71c3-43a1-81b4-30088f69768d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:14.258884 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d57aa9-71c3-43a1-81b4-30088f69768d None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:14.264855 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 60/117] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:14.264855 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d57aa9-71c3-43a1-81b4-30088f69768d None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:14.300230 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:14.352405 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5df602-4a17-4da0-ae95-2df7d3ed84a6 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:14.389946 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d57aa9-71c3-43a1-81b4-30088f69768d None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:14.394183 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377514c-eeac-4b4f-86cc-8901ad8f6949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:14.395080 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377514c-eeac-4b4f-86cc-8901ad8f6949 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:14.395436 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377514c-eeac-4b4f-86cc-8901ad8f6949 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:14.395785 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377514c-eeac-4b4f-86cc-8901ad8f6949 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:14.408811 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2377514c-eeac-4b4f-86cc-8901ad8f6949 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:14.418842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-2377514c-eeac-4b4f-86cc-8901ad8f6949 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 19:24:14.418842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:14.418842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:14.418842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:14.418842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:14.418842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:14.418842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:14.418842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:14.418842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:14.418842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:14.418842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:14.418842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:14.418842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:14.418842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:14.418842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:14.418842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:14.420198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:14.420198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:14.420198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:14.420198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:14.420198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:14.420198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:14.420198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:14.420198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:14.420198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:14.420198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:14.420198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:14.420198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:14.420198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:14.420198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:14.420198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:14.420198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:14.421054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:14.421054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:14.421054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:14.421054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:14.421054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:14.421054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:14.421054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:14.421054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:14.421054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:14.421054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:14.421054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 19:24:14.421054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:24:14.421054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 58/118] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:24:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:14.427205 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377514c-eeac-4b4f-86cc-8901ad8f6949 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:14.431923 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc3ca0-e378-4e2a-b20d-44aa7fad0923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:14.432818 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc3ca0-e378-4e2a-b20d-44aa7fad0923 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:14.433193 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc3ca0-e378-4e2a-b20d-44aa7fad0923 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:14.433542 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc3ca0-e378-4e2a-b20d-44aa7fad0923 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:14.448657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bc3ca0-e378-4e2a-b20d-44aa7fad0923 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:14.460123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-a1bc3ca0-e378-4e2a-b20d-44aa7fad0923 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:24:14.460123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:14.460123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:14.460123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:14.460123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:14.460123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:14.460123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:14.460123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:14.460123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:14.460123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:14.460123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:14.460123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:14.460123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:14.460123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:14.460123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:14.460123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:14.461187 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:14.461187 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:14.461187 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:14.461187 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:14.461187 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:14.461187 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:14.461187 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:14.461187 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:14.461187 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:14.461187 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:14.461187 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:14.461187 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:14.461187 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:14.461187 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:14.461187 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:14.461187 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:14.462167 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:14.462167 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:14.462167 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:14.462167 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:14.462167 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:14.462167 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:14.462167 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:14.462167 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:14.462167 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:14.462167 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:14.462167 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:24:14.462167 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:24:14.463459 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 61/119] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jun 15 19:24:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:14.463882 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbade98e-691d-418c-9762-8d3464090f32 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:14.466995 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbade98e-691d-418c-9762-8d3464090f32 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:14.466995 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbade98e-691d-418c-9762-8d3464090f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:14.486709 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-dbade98e-691d-418c-9762-8d3464090f32 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:14.488275 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-dbade98e-691d-418c-9762-8d3464090f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:14.549505 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:14.553738 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377514c-eeac-4b4f-86cc-8901ad8f6949 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:14.590181 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 62/120] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:14.591264 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbade98e-691d-418c-9762-8d3464090f32 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:14.602435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd1582-df34-4b80-8c87-0b0e82357584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:14.603110 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd1582-df34-4b80-8c87-0b0e82357584 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:14.603394 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd1582-df34-4b80-8c87-0b0e82357584 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:14.603721 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd1582-df34-4b80-8c87-0b0e82357584 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:14.604484 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dd1582-df34-4b80-8c87-0b0e82357584 None admin] RBAC: Adding query filter params (name=member) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:14.604790 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dd1582-df34-4b80-8c87-0b0e82357584 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:14.606215 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dd1582-df34-4b80-8c87-0b0e82357584 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:14.618761 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 59/121] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jun 15 19:24:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:14.619595 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd1582-df34-4b80-8c87-0b0e82357584 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:14.631834 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:14.730419 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd1582-df34-4b80-8c87-0b0e82357584 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:14.734378 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbade98e-691d-418c-9762-8d3464090f32 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:14.763961 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf0aad-10a7-49c1-97c4-10faa8fbb4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:14.764732 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf0aad-10a7-49c1-97c4-10faa8fbb4a2 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:14.765021 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf0aad-10a7-49c1-97c4-10faa8fbb4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:14.765300 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf0aad-10a7-49c1-97c4-10faa8fbb4a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:14.766054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cf0aad-10a7-49c1-97c4-10faa8fbb4a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:14.766391 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cf0aad-10a7-49c1-97c4-10faa8fbb4a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:14.767973 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cf0aad-10a7-49c1-97c4-10faa8fbb4a2 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:14.777032 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 60/122] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:24:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:14.777590 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf0aad-10a7-49c1-97c4-10faa8fbb4a2 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:14.779406 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84879203-aa49-4ab3-91ac-d409fd9be5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:14.780136 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84879203-aa49-4ab3-91ac-d409fd9be5fd None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:14.780422 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84879203-aa49-4ab3-91ac-d409fd9be5fd None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:14.780919 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84879203-aa49-4ab3-91ac-d409fd9be5fd None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:14.782522 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84879203-aa49-4ab3-91ac-d409fd9be5fd None admin] RBAC: Adding query filter params (name=member) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:14.782839 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84879203-aa49-4ab3-91ac-d409fd9be5fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:14.784241 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84879203-aa49-4ab3-91ac-d409fd9be5fd None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:14.795391 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 63/123] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jun 15 19:24:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:14.796226 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84879203-aa49-4ab3-91ac-d409fd9be5fd None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:14.830581 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:14.887726 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf0aad-10a7-49c1-97c4-10faa8fbb4a2 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:14.895757 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:14.914371 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84879203-aa49-4ab3-91ac-d409fd9be5fd None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:14.928899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af4c9b3-0082-4df3-aa00-f212a4a58a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:14.930063 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4c9b3-0082-4df3-aa00-f212a4a58a0e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:14.930559 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4c9b3-0082-4df3-aa00-f212a4a58a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:14.930975 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4c9b3-0082-4df3-aa00-f212a4a58a0e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:14.942313 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af4c9b3-0082-4df3-aa00-f212a4a58a0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:14.951088 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-4af4c9b3-0082-4df3-aa00-f212a4a58a0e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 19:24:14.951088 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:14.951088 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:14.951088 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:14.951088 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:14.951088 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:14.951088 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:14.951088 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:14.951088 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:14.951088 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:14.951088 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:14.951088 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:14.951088 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:14.951088 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:14.951088 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:14.951088 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:14.952139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:14.952139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:14.952139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:14.952139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:14.952139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:14.952139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:14.952139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:14.952139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:14.952139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:14.952139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:14.952139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:14.952139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:14.952139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:14.952139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:14.952139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:14.952139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:14.953036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:14.953036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:14.953036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:14.953036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:14.953036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:14.953036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:14.953036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:14.953036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:14.953036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:14.953036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:14.953036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 19:24:14.953036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:24:14.953610 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d501cb-0b84-4974-a481-60f16cf9d5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:14.953610 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d501cb-0b84-4974-a481-60f16cf9d5bb None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:14.953610 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d501cb-0b84-4974-a481-60f16cf9d5bb None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:14.953610 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d501cb-0b84-4974-a481-60f16cf9d5bb None admin] PATH_INFO: `/v3/users/abb9d50717324682b8fc92f386605834` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:14.954582 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d501cb-0b84-4974-a481-60f16cf9d5bb None admin] RBAC: Authorizing `identity:get_user(user_id=abb9d50717324682b8fc92f386605834)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:14.955337 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 61/124] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jun 15 19:24:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:14.956278 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af4c9b3-0082-4df3-aa00-f212a4a58a0e None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:14.961427 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d501cb-0b84-4974-a481-60f16cf9d5bb None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:14.963343 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 64/125] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:24:14 2022] GET /identity/v3/users/abb9d50717324682b8fc92f386605834 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:14.968585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d501cb-0b84-4974-a481-60f16cf9d5bb None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:15.051681 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:15.069996 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af4c9b3-0082-4df3-aa00-f212a4a58a0e None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:15.074774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d501cb-0b84-4974-a481-60f16cf9d5bb None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:15.105980 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b6ee54-c819-4865-837e-5288b9ad62c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:15.107701 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b6ee54-c819-4865-837e-5288b9ad62c7 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:15.108130 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b6ee54-c819-4865-837e-5288b9ad62c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:15.108562 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b6ee54-c819-4865-837e-5288b9ad62c7 None admin] PATH_INFO: `/v3/users/d018cc34dbeb4d56a3052de5c20b961c` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:15.109911 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b6ee54-c819-4865-837e-5288b9ad62c7 None admin] RBAC: Authorizing `identity:get_user(user_id=d018cc34dbeb4d56a3052de5c20b961c)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:15.112571 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b6ee54-c819-4865-837e-5288b9ad62c7 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:15.112700 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c87501-a883-41cc-b23f-7562d9eff466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:15.113526 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c87501-a883-41cc-b23f-7562d9eff466 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:15.113817 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c87501-a883-41cc-b23f-7562d9eff466 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:15.114115 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c87501-a883-41cc-b23f-7562d9eff466 None admin] PATH_INFO: `/v3/users/abb9d50717324682b8fc92f386605834` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:15.115257 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 62/126] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:24:14 2022] GET /identity/v3/users/d018cc34dbeb4d56a3052de5c20b961c => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:15.115360 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c87501-a883-41cc-b23f-7562d9eff466 None admin] RBAC: Authorizing `identity:get_user(user_id=abb9d50717324682b8fc92f386605834)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:15.116372 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b6ee54-c819-4865-837e-5288b9ad62c7 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:15.116896 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c87501-a883-41cc-b23f-7562d9eff466 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:15.118565 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 65/127] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:24:14 2022] GET /identity/v3/users/abb9d50717324682b8fc92f386605834 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:15.119495 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c87501-a883-41cc-b23f-7562d9eff466 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:15.145615 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:15.239150 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b6ee54-c819-4865-837e-5288b9ad62c7 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:15.242363 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c87501-a883-41cc-b23f-7562d9eff466 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:15.279959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de4424-7225-413b-8096-ac5f64488321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:15.280952 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de4424-7225-413b-8096-ac5f64488321 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:15.281431 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de4424-7225-413b-8096-ac5f64488321 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:15.281792 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450014ba-be42-43d8-a80c-9a7f2fcab779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:15.282190 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de4424-7225-413b-8096-ac5f64488321 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:15.282695 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450014ba-be42-43d8-a80c-9a7f2fcab779 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:15.282986 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450014ba-be42-43d8-a80c-9a7f2fcab779 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:15.283270 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450014ba-be42-43d8-a80c-9a7f2fcab779 None admin] PATH_INFO: `/v3/projects/0c11dde50af14022b1a1fe4e7a2f8a94` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:15.283733 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de4424-7225-413b-8096-ac5f64488321 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:15.284269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de4424-7225-413b-8096-ac5f64488321 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:15.284620 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450014ba-be42-43d8-a80c-9a7f2fcab779 None admin] RBAC: Authorizing `identity:get_project(project_id=0c11dde50af14022b1a1fe4e7a2f8a94)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:15.285938 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de4424-7225-413b-8096-ac5f64488321 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:15.286285 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450014ba-be42-43d8-a80c-9a7f2fcab779 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:15.288253 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 66/128] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:15 2022] GET /identity/v3/projects/0c11dde50af14022b1a1fe4e7a2f8a94 => generated 312 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:15.289272 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450014ba-be42-43d8-a80c-9a7f2fcab779 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:15.295345 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 63/129] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:24:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:15.296427 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de4424-7225-413b-8096-ac5f64488321 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:15.313962 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:15.398658 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:15.409847 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de4424-7225-413b-8096-ac5f64488321 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:15.414937 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450014ba-be42-43d8-a80c-9a7f2fcab779 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:15.444804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d23d796-ffe3-46fd-a6c5-c8e841549ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:15.445915 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23d796-ffe3-46fd-a6c5-c8e841549ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:15.446400 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23d796-ffe3-46fd-a6c5-c8e841549ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:15.446859 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23d796-ffe3-46fd-a6c5-c8e841549ea5 None admin] PATH_INFO: `/v3/projects/c4f989e5fad14bcebae5e279f22ee21d` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:15.448316 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d23d796-ffe3-46fd-a6c5-c8e841549ea5 None admin] RBAC: Authorizing `identity:get_project(project_id=c4f989e5fad14bcebae5e279f22ee21d)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:15.449929 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d23d796-ffe3-46fd-a6c5-c8e841549ea5 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:15.452432 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 64/130] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:15 2022] GET /identity/v3/projects/c4f989e5fad14bcebae5e279f22ee21d => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:15.453377 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d23d796-ffe3-46fd-a6c5-c8e841549ea5 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:15.453588 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689ceeef-2b08-4694-b4d8-4891e5a3967a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:15.454506 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ceeef-2b08-4694-b4d8-4891e5a3967a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:15.454782 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ceeef-2b08-4694-b4d8-4891e5a3967a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:15.455066 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ceeef-2b08-4694-b4d8-4891e5a3967a None admin] PATH_INFO: `/v3/projects/c4f989e5fad14bcebae5e279f22ee21d` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:15.456276 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689ceeef-2b08-4694-b4d8-4891e5a3967a None admin] RBAC: Authorizing `identity:get_project(project_id=c4f989e5fad14bcebae5e279f22ee21d)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:15.458786 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689ceeef-2b08-4694-b4d8-4891e5a3967a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:15.460345 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 67/131] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:15 2022] GET /identity/v3/projects/c4f989e5fad14bcebae5e279f22ee21d => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:15.461284 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689ceeef-2b08-4694-b4d8-4891e5a3967a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:15.538692 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:15.558470 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d23d796-ffe3-46fd-a6c5-c8e841549ea5 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:15.574843 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689ceeef-2b08-4694-b4d8-4891e5a3967a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:15.596417 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a586c9c-5d71-4d63-9945-23e4dc837bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:15.597331 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a586c9c-5d71-4d63-9945-23e4dc837bf8 None admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:15.597620 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a586c9c-5d71-4d63-9945-23e4dc837bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:15.597905 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a586c9c-5d71-4d63-9945-23e4dc837bf8 None admin] PATH_INFO: `/v3/projects/0c11dde50af14022b1a1fe4e7a2f8a94/users/abb9d50717324682b8fc92f386605834/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:15.600566 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a586c9c-5d71-4d63-9945-23e4dc837bf8 None admin] RBAC: Authorizing `identity:create_grant(project_id=0c11dde50af14022b1a1fe4e7a2f8a94, user_id=abb9d50717324682b8fc92f386605834, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:15.602633 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a586c9c-5d71-4d63-9945-23e4dc837bf8 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:15.618576 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38c17f-dd61-4584-8adf-493c894b1e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:15.619841 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38c17f-dd61-4584-8adf-493c894b1e24 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:15.620130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38c17f-dd61-4584-8adf-493c894b1e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:15.620414 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38c17f-dd61-4584-8adf-493c894b1e24 None admin] PATH_INFO: `/v3/users/abb9d50717324682b8fc92f386605834` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:15.624142 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b38c17f-dd61-4584-8adf-493c894b1e24 None admin] RBAC: Authorizing `identity:get_user(user_id=abb9d50717324682b8fc92f386605834)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:15.626370 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 65/132] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jun 15 19:24:15 2022] PUT /identity/v3/projects/0c11dde50af14022b1a1fe4e7a2f8a94/users/abb9d50717324682b8fc92f386605834/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:24:15.626777 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b38c17f-dd61-4584-8adf-493c894b1e24 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:15.627170 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a586c9c-5d71-4d63-9945-23e4dc837bf8 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:15.629225 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 68/133] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:24:15 2022] GET /identity/v3/users/abb9d50717324682b8fc92f386605834 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:15.630304 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38c17f-dd61-4584-8adf-493c894b1e24 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:15.670306 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:15.755394 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38c17f-dd61-4584-8adf-493c894b1e24 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:15.794526 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec3e192-ba5c-4320-bb9e-bef5da08094a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:15.795557 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3e192-ba5c-4320-bb9e-bef5da08094a None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:15.795848 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3e192-ba5c-4320-bb9e-bef5da08094a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:15.796180 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3e192-ba5c-4320-bb9e-bef5da08094a None admin] PATH_INFO: `/v3/projects/c4f989e5fad14bcebae5e279f22ee21d/users/d018cc34dbeb4d56a3052de5c20b961c/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:15.798585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec3e192-ba5c-4320-bb9e-bef5da08094a None admin] RBAC: Authorizing `identity:create_grant(project_id=c4f989e5fad14bcebae5e279f22ee21d, user_id=d018cc34dbeb4d56a3052de5c20b961c, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:15.800131 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec3e192-ba5c-4320-bb9e-bef5da08094a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:15.821213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 69/134] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jun 15 19:24:15 2022] PUT /identity/v3/projects/c4f989e5fad14bcebae5e279f22ee21d/users/d018cc34dbeb4d56a3052de5c20b961c/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:24:15.822266 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a586c9c-5d71-4d63-9945-23e4dc837bf8 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:15.824326 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec3e192-ba5c-4320-bb9e-bef5da08094a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:15.905696 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8623191-96e9-47c2-83c7-e796fb0a46bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:15.906665 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8623191-96e9-47c2-83c7-e796fb0a46bb None admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:15.906954 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8623191-96e9-47c2-83c7-e796fb0a46bb None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:15.907237 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8623191-96e9-47c2-83c7-e796fb0a46bb None admin] PATH_INFO: `/v3/projects/c4f989e5fad14bcebae5e279f22ee21d/users/abb9d50717324682b8fc92f386605834/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:15.914190 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8623191-96e9-47c2-83c7-e796fb0a46bb None admin] RBAC: Authorizing `identity:create_grant(project_id=c4f989e5fad14bcebae5e279f22ee21d, user_id=abb9d50717324682b8fc92f386605834, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:15.919625 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8623191-96e9-47c2-83c7-e796fb0a46bb None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:15.950515 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 66/135] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jun 15 19:24:15 2022] PUT /identity/v3/projects/c4f989e5fad14bcebae5e279f22ee21d/users/abb9d50717324682b8fc92f386605834/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:24:15.979581 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:15.999948 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec3e192-ba5c-4320-bb9e-bef5da08094a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:16.035689 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2cdc4-7755-4013-8239-f7f4da0802c8 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:16.035985 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2cdc4-7755-4013-8239-f7f4da0802c8 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:16.036261 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2cdc4-7755-4013-8239-f7f4da0802c8 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:16.038026 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 67/136] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 19:24:16.045045 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467bee0-fa9e-44dc-8944-ee08cd5644f1 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:16.045434 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467bee0-fa9e-44dc-8944-ee08cd5644f1 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:16.045719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467bee0-fa9e-44dc-8944-ee08cd5644f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:16.046062 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff1f99-4868-4153-906e-2f0c088cefa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:16.047043 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff1f99-4868-4153-906e-2f0c088cefa5 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:16.047337 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff1f99-4868-4153-906e-2f0c088cefa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:16.047723 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff1f99-4868-4153-906e-2f0c088cefa5 None admin] PATH_INFO: `/v3/projects/c4f989e5fad14bcebae5e279f22ee21d` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:16.051199 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff1f99-4868-4153-906e-2f0c088cefa5 None admin] RBAC: Authorizing `identity:get_project(project_id=c4f989e5fad14bcebae5e279f22ee21d)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:16.052849 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff1f99-4868-4153-906e-2f0c088cefa5 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:16.055259 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 70/137] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:15 2022] GET /identity/v3/projects/c4f989e5fad14bcebae5e279f22ee21d => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:16.058322 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:16.062648 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff1f99-4868-4153-906e-2f0c088cefa5 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:16.077204 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-c467bee0-fa9e-44dc-8944-ee08cd5644f1 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:16.080082 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-c467bee0-fa9e-44dc-8944-ee08cd5644f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:16.183372 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 68/138] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:16.195082 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff1f99-4868-4153-906e-2f0c088cefa5 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:16.211827 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c467bee0-fa9e-44dc-8944-ee08cd5644f1 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:16.245097 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d2a7c-21f1-474a-8c4d-8d51f81e9734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:16.246010 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d2a7c-21f1-474a-8c4d-8d51f81e9734 None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:16.246324 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d2a7c-21f1-474a-8c4d-8d51f81e9734 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:16.246608 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d2a7c-21f1-474a-8c4d-8d51f81e9734 None admin] PATH_INFO: `/v3/projects/c4f989e5fad14bcebae5e279f22ee21d/users/abb9d50717324682b8fc92f386605834/roles/3920d5126e22480ab9a4eec9cf0fd4e7` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:16.254075 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d2a7c-21f1-474a-8c4d-8d51f81e9734 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4f989e5fad14bcebae5e279f22ee21d, user_id=abb9d50717324682b8fc92f386605834, role_id=3920d5126e22480ab9a4eec9cf0fd4e7)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:16.255582 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d2a7c-21f1-474a-8c4d-8d51f81e9734 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:16.281367 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 71/139] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jun 15 19:24:16 2022] PUT /identity/v3/projects/c4f989e5fad14bcebae5e279f22ee21d/users/abb9d50717324682b8fc92f386605834/roles/3920d5126e22480ab9a4eec9cf0fd4e7 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:24:16.351901 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c467bee0-fa9e-44dc-8944-ee08cd5644f1 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:16.391073 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:16.394989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64976f2e-1175-4624-9726-5b8201593d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:16.396038 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64976f2e-1175-4624-9726-5b8201593d17 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:16.396540 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64976f2e-1175-4624-9726-5b8201593d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:16.397049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64976f2e-1175-4624-9726-5b8201593d17 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:16.398370 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64976f2e-1175-4624-9726-5b8201593d17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:16.399933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64976f2e-1175-4624-9726-5b8201593d17 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:16.401667 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 69/140] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:24:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:24:16.407588 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d2a7c-21f1-474a-8c4d-8d51f81e9734 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:16.437788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:16.573344 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d2a7c-21f1-474a-8c4d-8d51f81e9734 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:16.624111 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157de318-58d1-48f4-bdb9-8d9525a8a812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:16.625157 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157de318-58d1-48f4-bdb9-8d9525a8a812 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:16.625656 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157de318-58d1-48f4-bdb9-8d9525a8a812 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:16.626175 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157de318-58d1-48f4-bdb9-8d9525a8a812 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:16.627116 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157de318-58d1-48f4-bdb9-8d9525a8a812 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:16.628783 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157de318-58d1-48f4-bdb9-8d9525a8a812 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:16.640947 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa433fe9-ceaf-4931-9cfd-8f79677a57a1 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:16.641239 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa433fe9-ceaf-4931-9cfd-8f79677a57a1 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:16.641701 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa433fe9-ceaf-4931-9cfd-8f79677a57a1 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:16.645036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 70/141] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 19:24:16.650798 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f912c-8cd7-4c02-910a-3c79cfcd3578 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:16.651197 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f912c-8cd7-4c02-910a-3c79cfcd3578 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:16.651567 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f912c-8cd7-4c02-910a-3c79cfcd3578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:16.673838 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 72/142] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Jun 15 19:24:16 2022] POST /identity/v3/users => generated 283 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:24:16.684269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-915f912c-8cd7-4c02-910a-3c79cfcd3578 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:16.686370 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-915f912c-8cd7-4c02-910a-3c79cfcd3578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:16.791653 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 71/143] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 19:24:16.826767 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157de318-58d1-48f4-bdb9-8d9525a8a812 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:16.915886 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:16.960560 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157de318-58d1-48f4-bdb9-8d9525a8a812 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:17.006878 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12d6d57-0080-4ef1-b8df-5f9228a808e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:17.008008 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d6d57-0080-4ef1-b8df-5f9228a808e9 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:17.008468 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d6d57-0080-4ef1-b8df-5f9228a808e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:17.009192 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d6d57-0080-4ef1-b8df-5f9228a808e9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:17.023274 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12d6d57-0080-4ef1-b8df-5f9228a808e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:17.031434 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-f12d6d57-0080-4ef1-b8df-5f9228a808e9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 19:24:17.031434 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:17.031434 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:17.031434 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:17.031434 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:17.031434 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:17.031434 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:17.031434 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:17.031434 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:17.031434 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:17.031434 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:17.031434 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:17.031434 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:17.031434 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:17.031434 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:17.031434 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:17.032350 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:17.032350 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:17.032350 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:17.032350 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:17.032350 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:17.032350 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:17.032350 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:17.032350 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:17.032350 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:17.032350 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:17.032350 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:17.032350 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:17.032350 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:17.032350 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:17.032350 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:17.032350 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:17.033145 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:17.033145 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:17.033145 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:17.033145 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:17.033145 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:17.033145 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:17.033145 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:17.033145 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:17.033145 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:17.033145 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:17.033145 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 19:24:17.033145 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:24:17.035572 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 73/144] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:24:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:17.041000 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f912c-8cd7-4c02-910a-3c79cfcd3578 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:17.163385 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f912c-8cd7-4c02-910a-3c79cfcd3578 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:17.203261 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:17.210170 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbdb803-1474-477e-94c8-8fb3c76e0571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:17.211363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbdb803-1474-477e-94c8-8fb3c76e0571 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:17.211808 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbdb803-1474-477e-94c8-8fb3c76e0571 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:17.212316 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbdb803-1474-477e-94c8-8fb3c76e0571 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:17.213249 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbdb803-1474-477e-94c8-8fb3c76e0571 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:17.213710 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbdb803-1474-477e-94c8-8fb3c76e0571 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:17.215274 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbdb803-1474-477e-94c8-8fb3c76e0571 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:17.230153 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 72/145] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jun 15 19:24:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:17.235288 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12d6d57-0080-4ef1-b8df-5f9228a808e9 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:17.376975 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12d6d57-0080-4ef1-b8df-5f9228a808e9 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:17.410951 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:17.419955 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d47138-7485-4c6d-96ec-d7cf47847482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:17.421043 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d47138-7485-4c6d-96ec-d7cf47847482 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:17.421498 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d47138-7485-4c6d-96ec-d7cf47847482 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:17.421975 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d47138-7485-4c6d-96ec-d7cf47847482 None admin] PATH_INFO: `/v3/users/a1ad396a5e26438288c0d36a4fe00a05` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:17.423376 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d47138-7485-4c6d-96ec-d7cf47847482 None admin] RBAC: Authorizing `identity:get_user(user_id=a1ad396a5e26438288c0d36a4fe00a05)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:17.425005 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d47138-7485-4c6d-96ec-d7cf47847482 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:17.426944 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 74/146] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:24:17 2022] GET /identity/v3/users/a1ad396a5e26438288c0d36a4fe00a05 => generated 289 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:17.435236 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbdb803-1474-477e-94c8-8fb3c76e0571 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:17.570433 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbdb803-1474-477e-94c8-8fb3c76e0571 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:17.611876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b96134-6fdb-4e61-a0a0-485a40f0ff0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:17.612744 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b96134-6fdb-4e61-a0a0-485a40f0ff0a None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:17.613033 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b96134-6fdb-4e61-a0a0-485a40f0ff0a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:17.613312 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b96134-6fdb-4e61-a0a0-485a40f0ff0a None admin] PATH_INFO: `/v3/projects/c4f989e5fad14bcebae5e279f22ee21d` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:17.618877 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b96134-6fdb-4e61-a0a0-485a40f0ff0a None admin] RBAC: Authorizing `identity:get_project(project_id=c4f989e5fad14bcebae5e279f22ee21d)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:17.620311 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b96134-6fdb-4e61-a0a0-485a40f0ff0a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:17.622463 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 73/147] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:17 2022] GET /identity/v3/projects/c4f989e5fad14bcebae5e279f22ee21d => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:17.627774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d47138-7485-4c6d-96ec-d7cf47847482 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:17.756582 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d47138-7485-4c6d-96ec-d7cf47847482 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:17.793067 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9196b-8c60-41e6-b997-dfb8f1be3acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:17.794435 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9196b-8c60-41e6-b997-dfb8f1be3acc None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:17.794935 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9196b-8c60-41e6-b997-dfb8f1be3acc None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:17.795429 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9196b-8c60-41e6-b997-dfb8f1be3acc None admin] PATH_INFO: `/v3/projects/c4f989e5fad14bcebae5e279f22ee21d/users/a1ad396a5e26438288c0d36a4fe00a05/roles/4e60cc1f881a4caebdc208088b7ffcaa` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:17.797632 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e9196b-8c60-41e6-b997-dfb8f1be3acc None admin] RBAC: Authorizing `identity:create_grant(project_id=c4f989e5fad14bcebae5e279f22ee21d, user_id=a1ad396a5e26438288c0d36a4fe00a05, role_id=4e60cc1f881a4caebdc208088b7ffcaa)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:17.799376 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e9196b-8c60-41e6-b997-dfb8f1be3acc None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:17.818974 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 75/148] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jun 15 19:24:17 2022] PUT /identity/v3/projects/c4f989e5fad14bcebae5e279f22ee21d/users/a1ad396a5e26438288c0d36a4fe00a05/roles/4e60cc1f881a4caebdc208088b7ffcaa => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:24:19.256741 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928449cd-fd40-493b-945d-c818d5339513 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:19.257275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928449cd-fd40-493b-945d-c818d5339513 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:19.257275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928449cd-fd40-493b-945d-c818d5339513 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:19.259169 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 74/149] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:19.270646 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08edc075-ddff-44b4-81f1-a5dce7515236 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:19.271122 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08edc075-ddff-44b4-81f1-a5dce7515236 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:19.271210 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08edc075-ddff-44b4-81f1-a5dce7515236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:19.271410 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829a20a-844a-401c-83eb-8192010b503c None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:19.271635 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829a20a-844a-401c-83eb-8192010b503c None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:19.271916 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829a20a-844a-401c-83eb-8192010b503c None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:19.273401 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 75/150] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 19:24:19.279065 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e6fa4-aaaa-43c4-8381-2be6c35e2ba7 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:19.279357 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e6fa4-aaaa-43c4-8381-2be6c35e2ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:19.279638 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e6fa4-aaaa-43c4-8381-2be6c35e2ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:19.295376 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-08edc075-ddff-44b4-81f1-a5dce7515236 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:19.296583 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-08edc075-ddff-44b4-81f1-a5dce7515236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:19.305320 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-3e0e6fa4-aaaa-43c4-8381-2be6c35e2ba7 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:19.306654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-3e0e6fa4-aaaa-43c4-8381-2be6c35e2ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:19.383384 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:19.412381 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 76/151] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:19.426448 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:19.463818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 76/152] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:19.468535 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08edc075-ddff-44b4-81f1-a5dce7515236 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:19.521698 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e6fa4-aaaa-43c4-8381-2be6c35e2ba7 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:19.588405 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08edc075-ddff-44b4-81f1-a5dce7515236 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:19.629638 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82598de6-efcc-499e-a9a5-3d272cb1528d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:19.630476 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82598de6-efcc-499e-a9a5-3d272cb1528d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:19.630764 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82598de6-efcc-499e-a9a5-3d272cb1528d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:19.631038 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82598de6-efcc-499e-a9a5-3d272cb1528d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:19.651405 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82598de6-efcc-499e-a9a5-3d272cb1528d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:19.660156 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e6fa4-aaaa-43c4-8381-2be6c35e2ba7 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:19.660416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-82598de6-efcc-499e-a9a5-3d272cb1528d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:24:19.660416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:19.660416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:19.660416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:19.660416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:19.660416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:19.660416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:19.660416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:19.660416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:19.660416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:19.660416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:19.660416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:19.660416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:19.660416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:19.660416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:19.660416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:19.661217 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:19.661217 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:19.661217 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:19.661217 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:19.661217 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:19.661217 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:19.661217 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:19.661217 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:19.661217 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:19.661217 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:19.661217 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:19.661217 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:19.661217 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:19.661217 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:19.661217 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:19.661217 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:19.661972 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:19.661972 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:19.661972 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:19.661972 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:19.661972 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:19.661972 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:19.661972 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:19.661972 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:19.661972 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:19.661972 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:19.661972 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:24:19.661972 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:24:19.661972 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 77/153] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jun 15 19:24:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:19.668378 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82598de6-efcc-499e-a9a5-3d272cb1528d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:19.718865 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c8995b-3b29-4941-9f81-a9fe67689102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:19.719675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8995b-3b29-4941-9f81-a9fe67689102 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:19.719957 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8995b-3b29-4941-9f81-a9fe67689102 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:19.720227 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8995b-3b29-4941-9f81-a9fe67689102 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:19.731914 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c8995b-3b29-4941-9f81-a9fe67689102 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:19.739769 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-e1c8995b-3b29-4941-9f81-a9fe67689102 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 19:24:19.739769 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:19.739769 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:19.739769 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:19.739769 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:19.739769 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:19.739769 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:19.739769 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:19.739769 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:19.739769 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:19.739769 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:19.739769 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:19.739769 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:19.739769 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:19.739769 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:19.739769 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:19.741914 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:19.741914 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:19.741914 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:19.741914 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:19.741914 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:19.741914 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:19.741914 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:19.741914 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:19.741914 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:19.741914 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:19.741914 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:19.741914 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:19.741914 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:19.741914 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:19.741914 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:19.741914 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:19.743893 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:19.743893 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:19.743893 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:19.743893 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:19.743893 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:19.743893 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:19.743893 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:19.743893 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:19.743893 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:19.743893 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:19.743893 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 19:24:19.743893 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:24:19.743893 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 77/154] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:24:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:24:19.751385 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c8995b-3b29-4941-9f81-a9fe67689102 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:19.763211 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:19.805015 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:19.856761 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c8995b-3b29-4941-9f81-a9fe67689102 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:19.871004 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82598de6-efcc-499e-a9a5-3d272cb1528d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:19.915585 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a9e98-ba63-4ded-9c4c-f1d08e6c3d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:19.915585 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a9e98-ba63-4ded-9c4c-f1d08e6c3d88 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:19.916098 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a9e98-ba63-4ded-9c4c-f1d08e6c3d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:19.916098 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a9e98-ba63-4ded-9c4c-f1d08e6c3d88 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:19.916813 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a9e98-ba63-4ded-9c4c-f1d08e6c3d88 None admin] RBAC: Adding query filter params (name=member) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:19.917126 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a9e98-ba63-4ded-9c4c-f1d08e6c3d88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:19.918568 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a9e98-ba63-4ded-9c4c-f1d08e6c3d88 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:19.931801 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 78/155] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jun 15 19:24:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:24:19.933129 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f028848-3404-41bf-82b2-0311e96e3d3d None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:19.933421 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f028848-3404-41bf-82b2-0311e96e3d3d None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:19.933698 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f028848-3404-41bf-82b2-0311e96e3d3d None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:19.935286 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 79/156] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:19.936707 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac5a405-4b9a-46c6-b7a4-66c49d273c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:19.937668 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5a405-4b9a-46c6-b7a4-66c49d273c47 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:19.938230 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5a405-4b9a-46c6-b7a4-66c49d273c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:19.938754 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5a405-4b9a-46c6-b7a4-66c49d273c47 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:19.940801 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac5a405-4b9a-46c6-b7a4-66c49d273c47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:19.941317 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac5a405-4b9a-46c6-b7a4-66c49d273c47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:19.941430 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f028848-3404-41bf-82b2-0311e96e3d3d None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:19.943156 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac5a405-4b9a-46c6-b7a4-66c49d273c47 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:19.961079 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 78/157] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:24:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:24:19.962897 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eae831-f7f0-4cba-aaa5-b43fdc05cdc3 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:19.963445 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eae831-f7f0-4cba-aaa5-b43fdc05cdc3 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:19.963928 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eae831-f7f0-4cba-aaa5-b43fdc05cdc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:19.990154 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-86eae831-f7f0-4cba-aaa5-b43fdc05cdc3 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:19.991697 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-86eae831-f7f0-4cba-aaa5-b43fdc05cdc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:20.116034 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f028848-3404-41bf-82b2-0311e96e3d3d None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:20.142121 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 79/158] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:20.144475 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eae831-f7f0-4cba-aaa5-b43fdc05cdc3 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:20.147377 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:20.189246 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c8b0c-004d-45d3-9627-802229a59787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:20.190158 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c8b0c-004d-45d3-9627-802229a59787 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:20.190499 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c8b0c-004d-45d3-9627-802229a59787 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:20.190783 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c8b0c-004d-45d3-9627-802229a59787 None admin] PATH_INFO: `/v3/users/abb9d50717324682b8fc92f386605834` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:20.194810 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550c8b0c-004d-45d3-9627-802229a59787 None admin] RBAC: Authorizing `identity:get_user(user_id=abb9d50717324682b8fc92f386605834)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:20.196280 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550c8b0c-004d-45d3-9627-802229a59787 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:20.200907 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 80/159] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:24:19 2022] GET /identity/v3/users/abb9d50717324682b8fc92f386605834 => generated 275 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:20.204141 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c8b0c-004d-45d3-9627-802229a59787 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:20.243144 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:20.297065 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eae831-f7f0-4cba-aaa5-b43fdc05cdc3 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:20.365527 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d872352-c2be-4ec1-af31-e90b5c7710d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:20.366493 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d872352-c2be-4ec1-af31-e90b5c7710d5 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:20.366720 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d872352-c2be-4ec1-af31-e90b5c7710d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:20.366992 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d872352-c2be-4ec1-af31-e90b5c7710d5 None admin] PATH_INFO: `/v3/users/d018cc34dbeb4d56a3052de5c20b961c` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:20.368248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d872352-c2be-4ec1-af31-e90b5c7710d5 None admin] RBAC: Authorizing `identity:get_user(user_id=d018cc34dbeb4d56a3052de5c20b961c)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:20.369739 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d872352-c2be-4ec1-af31-e90b5c7710d5 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:20.371436 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 80/160] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:24:20 2022] GET /identity/v3/users/d018cc34dbeb4d56a3052de5c20b961c => generated 247 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:20.373769 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d872352-c2be-4ec1-af31-e90b5c7710d5 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:20.374877 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c8b0c-004d-45d3-9627-802229a59787 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:20.409421 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56692aa4-1be5-4868-9d41-9c060299c668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:20.410077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56692aa4-1be5-4868-9d41-9c060299c668 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:20.410391 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56692aa4-1be5-4868-9d41-9c060299c668 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:20.410669 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56692aa4-1be5-4868-9d41-9c060299c668 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:20.490444 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:20.545201 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56692aa4-1be5-4868-9d41-9c060299c668 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:20.545201 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56692aa4-1be5-4868-9d41-9c060299c668 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:20.579579 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d872352-c2be-4ec1-af31-e90b5c7710d5 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:20.594419 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:20.663695 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001f691-6831-4014-8f97-df5df3b56917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:20.664413 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001f691-6831-4014-8f97-df5df3b56917 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:20.664695 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001f691-6831-4014-8f97-df5df3b56917 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:20.664975 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001f691-6831-4014-8f97-df5df3b56917 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:20.694846 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e001f691-6831-4014-8f97-df5df3b56917 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:20.702982 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-e001f691-6831-4014-8f97-df5df3b56917 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 19:24:20.702982 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:20.702982 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:20.702982 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:20.702982 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:20.702982 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:20.702982 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:20.702982 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:20.702982 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:20.702982 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:20.702982 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:20.702982 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:20.702982 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:20.702982 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:20.702982 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:20.702982 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:20.703867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:20.703867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:20.703867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:20.703867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:20.703867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:20.703867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:20.703867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:20.703867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:20.703867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:20.703867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:20.703867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:20.703867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:20.703867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:20.703867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:20.703867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:20.703867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:20.704637 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:20.704637 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:20.704637 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:20.704637 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:20.704637 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:20.704637 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:20.704637 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:20.704637 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:20.704637 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:20.704637 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:20.704637 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 19:24:20.704637 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:24:20.704637 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 81/161] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:24:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:20.705741 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001f691-6831-4014-8f97-df5df3b56917 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:20.733724 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 81/162] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:24:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 19:24:20.734753 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56692aa4-1be5-4868-9d41-9c060299c668 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:20.827533 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:20.854319 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56692aa4-1be5-4868-9d41-9c060299c668 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:20.894404 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d6a4c-c585-4a06-94f1-d094dc233cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:20.895271 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d6a4c-c585-4a06-94f1-d094dc233cab None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:20.895559 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d6a4c-c585-4a06-94f1-d094dc233cab None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:20.895876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d6a4c-c585-4a06-94f1-d094dc233cab None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:20.896634 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306d6a4c-c585-4a06-94f1-d094dc233cab None admin] RBAC: Adding query filter params (name=member) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:20.896940 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306d6a4c-c585-4a06-94f1-d094dc233cab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:20.898476 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306d6a4c-c585-4a06-94f1-d094dc233cab None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:20.908335 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 82/163] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jun 15 19:24:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:20.909237 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d6a4c-c585-4a06-94f1-d094dc233cab None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:21.019874 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d6a4c-c585-4a06-94f1-d094dc233cab None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:21.020097 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001f691-6831-4014-8f97-df5df3b56917 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:21.057566 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e92645-44ba-440a-9139-d36e0c7c1fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:21.058526 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e92645-44ba-440a-9139-d36e0c7c1fef None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:21.058858 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e92645-44ba-440a-9139-d36e0c7c1fef None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:21.060041 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e92645-44ba-440a-9139-d36e0c7c1fef None admin] PATH_INFO: `/v3/projects/0c11dde50af14022b1a1fe4e7a2f8a94` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:21.061405 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e92645-44ba-440a-9139-d36e0c7c1fef None admin] RBAC: Authorizing `identity:get_project(project_id=0c11dde50af14022b1a1fe4e7a2f8a94)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:21.062935 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e92645-44ba-440a-9139-d36e0c7c1fef None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:21.064983 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 83/164] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:20 2022] GET /identity/v3/projects/0c11dde50af14022b1a1fe4e7a2f8a94 => generated 312 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:21.066278 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06ea1f-f191-48a7-aac7-3da4cc5a1d4c None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:21.067025 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06ea1f-f191-48a7-aac7-3da4cc5a1d4c None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:21.067426 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06ea1f-f191-48a7-aac7-3da4cc5a1d4c None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:21.068675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 84/165] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:21.069432 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec06ea1f-f191-48a7-aac7-3da4cc5a1d4c None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:21.084369 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:21.123399 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e83d1-698c-4394-ba9e-b47e4f68c742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:21.124074 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e83d1-698c-4394-ba9e-b47e4f68c742 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:21.124369 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e83d1-698c-4394-ba9e-b47e4f68c742 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:21.124649 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e83d1-698c-4394-ba9e-b47e4f68c742 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:21.192441 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec06ea1f-f191-48a7-aac7-3da4cc5a1d4c None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:21.195200 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:21.232880 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e73b2da-87f1-4cbf-83f3-57171b337b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:21.233754 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e73b2da-87f1-4cbf-83f3-57171b337b18 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:21.234078 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e73b2da-87f1-4cbf-83f3-57171b337b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:21.234375 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e73b2da-87f1-4cbf-83f3-57171b337b18 None admin] PATH_INFO: `/v3/users/abb9d50717324682b8fc92f386605834` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:21.235795 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e73b2da-87f1-4cbf-83f3-57171b337b18 None admin] RBAC: Authorizing `identity:get_user(user_id=abb9d50717324682b8fc92f386605834)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:21.238002 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e73b2da-87f1-4cbf-83f3-57171b337b18 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:21.240114 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 85/166] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:24:21 2022] GET /identity/v3/users/abb9d50717324682b8fc92f386605834 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:21.241102 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e73b2da-87f1-4cbf-83f3-57171b337b18 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:21.259277 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e83d1-698c-4394-ba9e-b47e4f68c742 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:21.261053 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e83d1-698c-4394-ba9e-b47e4f68c742 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:21.343228 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:21.366202 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e73b2da-87f1-4cbf-83f3-57171b337b18 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:21.404089 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a33a062-c53d-48d8-a234-5fc17b2e5478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:21.404861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33a062-c53d-48d8-a234-5fc17b2e5478 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:21.405071 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33a062-c53d-48d8-a234-5fc17b2e5478 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:21.405384 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33a062-c53d-48d8-a234-5fc17b2e5478 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:21.406251 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a33a062-c53d-48d8-a234-5fc17b2e5478 None admin] RBAC: Adding query filter params (role.id=e53a78912e764f8387e0cf637bb86bee, scope.project.id=0c11dde50af14022b1a1fe4e7a2f8a94, user.id=abb9d50717324682b8fc92f386605834) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:21.406562 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a33a062-c53d-48d8-a234-5fc17b2e5478 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:21.408213 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a33a062-c53d-48d8-a234-5fc17b2e5478 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:21.414808 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 86/167] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:24:21 2022] GET /identity/v3/role_assignments?user.id=abb9d50717324682b8fc92f386605834&scope.project.id=0c11dde50af14022b1a1fe4e7a2f8a94&role.id=e53a78912e764f8387e0cf637bb86bee => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:21.416108 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e0fdd5-6cc0-47e5-9ecc-de416f7b3ad0 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:21.416472 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e0fdd5-6cc0-47e5-9ecc-de416f7b3ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:21.416757 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e0fdd5-6cc0-47e5-9ecc-de416f7b3ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:21.435934 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-24e0fdd5-6cc0-47e5-9ecc-de416f7b3ad0 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:21.437181 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-24e0fdd5-6cc0-47e5-9ecc-de416f7b3ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:21.561123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 87/168] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:21.562461 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e0fdd5-6cc0-47e5-9ecc-de416f7b3ad0 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:21.581872 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 82/169] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:24:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 877 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 19:24:21.589750 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e83d1-698c-4394-ba9e-b47e4f68c742 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:21.668914 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:21.714594 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e0fdd5-6cc0-47e5-9ecc-de416f7b3ad0 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:21.744430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:21.774833 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3153b1c3-8bb2-4228-9ec0-0685ea11945d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:21.775886 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153b1c3-8bb2-4228-9ec0-0685ea11945d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:21.776366 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153b1c3-8bb2-4228-9ec0-0685ea11945d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:21.776846 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153b1c3-8bb2-4228-9ec0-0685ea11945d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:21.785827 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e83d1-698c-4394-ba9e-b47e4f68c742 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:21.833150 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb26fe-909c-46cb-b064-220b1cc783ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:21.834023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb26fe-909c-46cb-b064-220b1cc783ad None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:21.834335 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb26fe-909c-46cb-b064-220b1cc783ad None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:21.834610 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb26fe-909c-46cb-b064-220b1cc783ad None admin] PATH_INFO: `/v3/projects/c4f989e5fad14bcebae5e279f22ee21d` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:21.835810 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebb26fe-909c-46cb-b064-220b1cc783ad None admin] RBAC: Authorizing `identity:get_project(project_id=c4f989e5fad14bcebae5e279f22ee21d)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:21.837865 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebb26fe-909c-46cb-b064-220b1cc783ad None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:21.840317 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 83/170] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:21 2022] GET /identity/v3/projects/c4f989e5fad14bcebae5e279f22ee21d => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:24:21.841145 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb26fe-909c-46cb-b064-220b1cc783ad None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:21.931708 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3153b1c3-8bb2-4228-9ec0-0685ea11945d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:21.933581 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3153b1c3-8bb2-4228-9ec0-0685ea11945d None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:21.960233 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb26fe-909c-46cb-b064-220b1cc783ad None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:21.998524 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070bcacb-40bf-4d40-b8c3-01429f797a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:21.999654 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070bcacb-40bf-4d40-b8c3-01429f797a26 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:21.999947 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070bcacb-40bf-4d40-b8c3-01429f797a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:22.000223 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070bcacb-40bf-4d40-b8c3-01429f797a26 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:22.006430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:22.014662 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070bcacb-40bf-4d40-b8c3-01429f797a26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:22.030565 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-070bcacb-40bf-4d40-b8c3-01429f797a26 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 19:24:22.030565 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:22.030565 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:22.030565 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:22.030565 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:22.030565 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:22.030565 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:22.030565 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:22.030565 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:22.030565 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:22.030565 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:22.030565 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:22.030565 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:22.030565 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:22.030565 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:22.030565 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:22.031528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:22.031528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:22.031528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:22.031528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:22.031528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:22.031528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:22.031528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:22.031528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:22.031528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:22.031528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:22.031528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:22.031528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:22.031528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:22.031528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:22.031528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:22.031528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:22.032412 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:22.032412 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:22.032412 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:22.032412 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:22.032412 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:22.032412 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:22.032412 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:22.032412 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:22.032412 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:22.032412 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:22.032412 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 19:24:22.032412 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:24:22.032412 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 84/171] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jun 15 19:24:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:22.033862 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a14c3-69e8-47a2-a70e-77e3ac8ff14e None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:22.033862 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a14c3-69e8-47a2-a70e-77e3ac8ff14e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:22.034041 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a14c3-69e8-47a2-a70e-77e3ac8ff14e None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:22.035547 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 85/172] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:22.036192 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a14c3-69e8-47a2-a70e-77e3ac8ff14e None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:22.076204 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:22.110628 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 88/173] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:24:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 19:24:22.112290 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd12895-3416-4240-8cc5-867c19bd9eb4 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:22.117357 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd12895-3416-4240-8cc5-867c19bd9eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:22.117723 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd12895-3416-4240-8cc5-867c19bd9eb4 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:22.119384 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 89/174] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:22.120039 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd12895-3416-4240-8cc5-867c19bd9eb4 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:22.165744 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a14c3-69e8-47a2-a70e-77e3ac8ff14e None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:22.204971 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87749350-3228-4ce6-a34b-16ae5b1e6386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:22.205945 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87749350-3228-4ce6-a34b-16ae5b1e6386 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:22.206271 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87749350-3228-4ce6-a34b-16ae5b1e6386 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:22.206544 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87749350-3228-4ce6-a34b-16ae5b1e6386 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:22.207764 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87749350-3228-4ce6-a34b-16ae5b1e6386 None admin] RBAC: Adding query filter params (role.id=4f736313c26d4f8db48bbb2f346e50e4, scope.project.id=c4f989e5fad14bcebae5e279f22ee21d, user.id=d018cc34dbeb4d56a3052de5c20b961c) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:22.208082 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87749350-3228-4ce6-a34b-16ae5b1e6386 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:22.209522 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87749350-3228-4ce6-a34b-16ae5b1e6386 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:22.216228 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 86/175] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:24:22 2022] GET /identity/v3/role_assignments?user.id=d018cc34dbeb4d56a3052de5c20b961c&scope.project.id=c4f989e5fad14bcebae5e279f22ee21d&role.id=4f736313c26d4f8db48bbb2f346e50e4 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:22.217396 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a86f7-5a5b-47e5-a5f8-191eae491417 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:22.217687 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a86f7-5a5b-47e5-a5f8-191eae491417 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:22.217958 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a86f7-5a5b-47e5-a5f8-191eae491417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:22.242382 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-022a86f7-5a5b-47e5-a5f8-191eae491417 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:22.243511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-022a86f7-5a5b-47e5-a5f8-191eae491417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:22.268738 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd12895-3416-4240-8cc5-867c19bd9eb4 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:22.312393 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:22.325893 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda8ff98-3457-45d8-871a-731cdd4b7be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:22.326872 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8ff98-3457-45d8-871a-731cdd4b7be0 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:22.327157 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8ff98-3457-45d8-871a-731cdd4b7be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:22.327435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8ff98-3457-45d8-871a-731cdd4b7be0 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:22.328220 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda8ff98-3457-45d8-871a-731cdd4b7be0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:22.328530 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda8ff98-3457-45d8-871a-731cdd4b7be0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:22.329950 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda8ff98-3457-45d8-871a-731cdd4b7be0 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:22.332497 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 87/176] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:22.333786 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df084373-e917-4c0e-951a-9551aaf0d46a None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:22.334078 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df084373-e917-4c0e-951a-9551aaf0d46a None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:22.334425 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df084373-e917-4c0e-951a-9551aaf0d46a None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:22.340043 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 88/177] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:22.340043 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df084373-e917-4c0e-951a-9551aaf0d46a None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:22.356974 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 90/178] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:24:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:22.361470 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0171312-6938-49fa-bc54-d0b340cf690d None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:22.361470 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0171312-6938-49fa-bc54-d0b340cf690d None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:22.361470 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0171312-6938-49fa-bc54-d0b340cf690d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:22.378073 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-b0171312-6938-49fa-bc54-d0b340cf690d None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:22.379319 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-b0171312-6938-49fa-bc54-d0b340cf690d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:22.408430 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:22.456214 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 91/179] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:22.457290 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef8608-b003-4f8e-8723-34290ec3909d None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:22.457580 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef8608-b003-4f8e-8723-34290ec3909d None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:22.457850 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef8608-b003-4f8e-8723-34290ec3909d None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:22.459314 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 92/180] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:22.460219 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2547e3-1ed2-403d-994c-7bf7948727bf None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:22.460498 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2547e3-1ed2-403d-994c-7bf7948727bf None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:22.460768 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2547e3-1ed2-403d-994c-7bf7948727bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:22.479086 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df084373-e917-4c0e-951a-9551aaf0d46a None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:22.482893 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-3c2547e3-1ed2-403d-994c-7bf7948727bf None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:22.482893 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-3c2547e3-1ed2-403d-994c-7bf7948727bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:22.513752 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18174862-e871-446c-8289-618480a95053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:22.514619 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18174862-e871-446c-8289-618480a95053 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:22.514902 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18174862-e871-446c-8289-618480a95053 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:22.515191 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18174862-e871-446c-8289-618480a95053 None admin] PATH_INFO: `/v3/projects/c4f989e5fad14bcebae5e279f22ee21d` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:22.516482 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18174862-e871-446c-8289-618480a95053 None admin] RBAC: Authorizing `identity:get_project(project_id=c4f989e5fad14bcebae5e279f22ee21d)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:22.517804 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18174862-e871-446c-8289-618480a95053 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:22.519533 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 89/181] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:22 2022] GET /identity/v3/projects/c4f989e5fad14bcebae5e279f22ee21d => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:22.524357 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18174862-e871-446c-8289-618480a95053 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:22.549175 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 93/182] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:22.550217 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2547e3-1ed2-403d-994c-7bf7948727bf None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:22.590144 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:22.633609 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18174862-e871-446c-8289-618480a95053 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:22.641303 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:22.657054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2547e3-1ed2-403d-994c-7bf7948727bf None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:22.667990 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602ed26-dcc6-4f19-804a-4e9da9603bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:22.668877 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602ed26-dcc6-4f19-804a-4e9da9603bd7 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:22.669163 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602ed26-dcc6-4f19-804a-4e9da9603bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:22.669435 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602ed26-dcc6-4f19-804a-4e9da9603bd7 None admin] PATH_INFO: `/v3/users/abb9d50717324682b8fc92f386605834` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:22.670672 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9602ed26-dcc6-4f19-804a-4e9da9603bd7 None admin] RBAC: Authorizing `identity:get_user(user_id=abb9d50717324682b8fc92f386605834)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:22.672117 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9602ed26-dcc6-4f19-804a-4e9da9603bd7 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:22.673753 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 90/183] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:24:22 2022] GET /identity/v3/users/abb9d50717324682b8fc92f386605834 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:22.675001 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7115800-df5e-4fa3-bfa3-8d834dce094e None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:22.675292 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7115800-df5e-4fa3-bfa3-8d834dce094e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:22.675568 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7115800-df5e-4fa3-bfa3-8d834dce094e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:22.691170 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-f7115800-df5e-4fa3-bfa3-8d834dce094e None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:22.692339 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f7115800-df5e-4fa3-bfa3-8d834dce094e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:22.695556 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145716e3-8817-4c89-9c38-2b8f866f9779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:22.696354 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145716e3-8817-4c89-9c38-2b8f866f9779 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:22.696639 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145716e3-8817-4c89-9c38-2b8f866f9779 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:22.696917 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145716e3-8817-4c89-9c38-2b8f866f9779 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:22.708087 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145716e3-8817-4c89-9c38-2b8f866f9779 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:22.716337 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-145716e3-8817-4c89-9c38-2b8f866f9779 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:24:22.716337 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:22.716337 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:22.716337 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:22.716337 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:22.716337 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:22.716337 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:22.716337 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:22.716337 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:22.716337 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:22.716337 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:22.716337 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:22.716337 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:22.716337 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:22.716337 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:22.716337 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:22.717201 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:22.717201 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:22.717201 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:22.717201 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:22.717201 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:22.717201 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:22.717201 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:22.717201 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:22.717201 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:22.717201 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:22.717201 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:22.717201 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:22.717201 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:22.717201 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:22.717201 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:22.717201 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:22.718005 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:22.718005 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:22.718005 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:22.718005 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:22.718005 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:22.718005 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:22.718005 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:22.718005 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:22.718005 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:22.718005 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:22.718005 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:24:22.718005 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:24:22.718005 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 94/184] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jun 15 19:24:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:22.718701 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145716e3-8817-4c89-9c38-2b8f866f9779 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:22.761907 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 91/185] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:22.768970 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7115800-df5e-4fa3-bfa3-8d834dce094e None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:22.818380 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:22.824225 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145716e3-8817-4c89-9c38-2b8f866f9779 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:22.857318 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f703813-f111-494c-8d7d-46e86252789c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:22.858144 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f703813-f111-494c-8d7d-46e86252789c None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:22.858433 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f703813-f111-494c-8d7d-46e86252789c None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:22.858711 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f703813-f111-494c-8d7d-46e86252789c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:22.862649 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7115800-df5e-4fa3-bfa3-8d834dce094e None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:22.868722 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f703813-f111-494c-8d7d-46e86252789c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:22.876804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-4f703813-f111-494c-8d7d-46e86252789c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 19:24:22.876804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:22.876804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:22.876804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:22.876804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:22.876804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:22.876804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:22.876804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:22.876804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:22.876804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:22.876804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:22.876804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:22.876804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:22.876804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:22.876804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:22.876804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:22.877679 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:22.877679 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:22.877679 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:22.877679 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:22.877679 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:22.877679 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:22.877679 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:22.877679 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:22.877679 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:22.877679 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:22.877679 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:22.877679 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:22.877679 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:22.877679 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:22.877679 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:22.877679 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:22.878511 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:22.878511 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:22.878511 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:22.878511 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:22.878511 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:22.878511 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:22.878511 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:22.878511 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:22.878511 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:22.878511 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:22.878511 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 19:24:22.878511 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:24:22.878511 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 95/186] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jun 15 19:24:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:22.879176 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f703813-f111-494c-8d7d-46e86252789c None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:22.895796 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:22.900686 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b612c-168d-4d45-b2ba-997b3ffeb86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:22.901338 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b612c-168d-4d45-b2ba-997b3ffeb86a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:22.901621 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b612c-168d-4d45-b2ba-997b3ffeb86a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:22.901893 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b612c-168d-4d45-b2ba-997b3ffeb86a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:22.902765 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847b612c-168d-4d45-b2ba-997b3ffeb86a None admin] RBAC: Adding query filter params (role.id=e53a78912e764f8387e0cf637bb86bee, scope.project.id=c4f989e5fad14bcebae5e279f22ee21d, user.id=abb9d50717324682b8fc92f386605834) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:22.903691 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847b612c-168d-4d45-b2ba-997b3ffeb86a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:22.905139 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847b612c-168d-4d45-b2ba-997b3ffeb86a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:22.910949 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 92/187] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:24:22 2022] GET /identity/v3/role_assignments?user.id=abb9d50717324682b8fc92f386605834&scope.project.id=c4f989e5fad14bcebae5e279f22ee21d&role.id=e53a78912e764f8387e0cf637bb86bee => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:22.918422 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b612c-168d-4d45-b2ba-997b3ffeb86a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:23.003677 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f703813-f111-494c-8d7d-46e86252789c None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:23.027848 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b612c-168d-4d45-b2ba-997b3ffeb86a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:23.041159 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d9aa3-d629-432a-9545-00a263e3e308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:23.042060 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d9aa3-d629-432a-9545-00a263e3e308 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:23.042462 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d9aa3-d629-432a-9545-00a263e3e308 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:23.042804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d9aa3-d629-432a-9545-00a263e3e308 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:23.050376 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:23.057107 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459d9aa3-d629-432a-9545-00a263e3e308 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:23.065921 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-459d9aa3-d629-432a-9545-00a263e3e308 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 19:24:23.065921 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:23.065921 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:23.065921 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:23.065921 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:23.065921 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:23.065921 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:23.065921 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:23.065921 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:23.065921 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:23.065921 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:23.065921 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:23.065921 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:23.065921 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:23.065921 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:23.065921 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:23.067046 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:23.067046 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:23.067046 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:23.067046 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:23.067046 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:23.067046 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:23.067046 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:23.067046 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:23.067046 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:23.067046 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:23.067046 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:23.067046 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:23.067046 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:23.067046 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:23.067046 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:23.067046 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:23.067849 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:23.067849 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:23.067849 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:23.067849 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:23.067849 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:23.067849 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:23.067849 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:23.067849 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:23.067849 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:23.067849 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:23.067849 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 19:24:23.067849 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:24:23.067849 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 96/188] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:24:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:23.068586 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d9aa3-d629-432a-9545-00a263e3e308 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:23.074074 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0d283-6f2f-40b7-beba-269a86efe01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:23.074764 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0d283-6f2f-40b7-beba-269a86efe01e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:23.075151 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0d283-6f2f-40b7-beba-269a86efe01e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:23.075430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0d283-6f2f-40b7-beba-269a86efe01e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:23.155650 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa0d283-6f2f-40b7-beba-269a86efe01e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:23.157150 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa0d283-6f2f-40b7-beba-269a86efe01e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:23.162456 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:23.190827 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d9aa3-d629-432a-9545-00a263e3e308 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:23.223851 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ee0a3-9de1-4005-b709-dfff579bdf56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:23.224515 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ee0a3-9de1-4005-b709-dfff579bdf56 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:23.224802 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ee0a3-9de1-4005-b709-dfff579bdf56 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:23.225077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ee0a3-9de1-4005-b709-dfff579bdf56 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:23.225823 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66ee0a3-9de1-4005-b709-dfff579bdf56 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:23.226154 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66ee0a3-9de1-4005-b709-dfff579bdf56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:23.227565 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66ee0a3-9de1-4005-b709-dfff579bdf56 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:23.236010 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 97/189] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:24:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:23.240607 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ee0a3-9de1-4005-b709-dfff579bdf56 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:23.259837 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 93/190] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:24:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 19:24:23.260833 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0d283-6f2f-40b7-beba-269a86efe01e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:23.280449 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:23.342319 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ee0a3-9de1-4005-b709-dfff579bdf56 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:23.365685 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0d283-6f2f-40b7-beba-269a86efe01e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:23.376332 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04bf25-ea2d-482b-8cc2-986edb056602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:23.377118 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04bf25-ea2d-482b-8cc2-986edb056602 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:23.377398 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04bf25-ea2d-482b-8cc2-986edb056602 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:23.377671 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04bf25-ea2d-482b-8cc2-986edb056602 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:23.378855 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d04bf25-ea2d-482b-8cc2-986edb056602 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:23.380278 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d04bf25-ea2d-482b-8cc2-986edb056602 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:23.381876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 98/191] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:24:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:23.382661 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04bf25-ea2d-482b-8cc2-986edb056602 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:23.386607 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:23.404975 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca690d-5de3-4f0d-8f86-13d61daca963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:23.405650 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca690d-5de3-4f0d-8f86-13d61daca963 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:23.405935 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca690d-5de3-4f0d-8f86-13d61daca963 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:23.406264 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca690d-5de3-4f0d-8f86-13d61daca963 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:23.407023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca690d-5de3-4f0d-8f86-13d61daca963 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:23.407333 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca690d-5de3-4f0d-8f86-13d61daca963 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:23.408710 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca690d-5de3-4f0d-8f86-13d61daca963 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:23.416564 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 94/192] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:24:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:23.421152 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca690d-5de3-4f0d-8f86-13d61daca963 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:23.485387 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04bf25-ea2d-482b-8cc2-986edb056602 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:23.495736 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:23.515418 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca690d-5de3-4f0d-8f86-13d61daca963 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:23.518666 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c3cc11-1f40-4ed1-bbe2-9bbe821b6619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:23.519322 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c3cc11-1f40-4ed1-bbe2-9bbe821b6619 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:23.519601 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c3cc11-1f40-4ed1-bbe2-9bbe821b6619 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:23.519871 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c3cc11-1f40-4ed1-bbe2-9bbe821b6619 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:23.520608 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c3cc11-1f40-4ed1-bbe2-9bbe821b6619 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:23.520913 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c3cc11-1f40-4ed1-bbe2-9bbe821b6619 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:23.522308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c3cc11-1f40-4ed1-bbe2-9bbe821b6619 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:23.530290 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 99/193] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jun 15 19:24:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:23.531739 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c3cc11-1f40-4ed1-bbe2-9bbe821b6619 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:23.548847 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68fd0d-d282-4995-9941-d9434acf4cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:23.549801 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68fd0d-d282-4995-9941-d9434acf4cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:23.550189 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68fd0d-d282-4995-9941-d9434acf4cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:23.550472 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68fd0d-d282-4995-9941-d9434acf4cc7 None admin] PATH_INFO: `/v3/users/1041c9fb868f414798a4a50b20e459af` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:23.564224 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba68fd0d-d282-4995-9941-d9434acf4cc7 None admin] RBAC: Authorizing `identity:get_user(user_id=1041c9fb868f414798a4a50b20e459af)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:23.565693 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba68fd0d-d282-4995-9941-d9434acf4cc7 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:23.571522 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 95/194] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:24:23 2022] GET /identity/v3/users/1041c9fb868f414798a4a50b20e459af => generated 283 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:23.571522 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68fd0d-d282-4995-9941-d9434acf4cc7 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:23.596096 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:23.630840 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c3cc11-1f40-4ed1-bbe2-9bbe821b6619 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:23.665862 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7239ae-8e9b-4654-8945-0c316e922eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:23.666750 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7239ae-8e9b-4654-8945-0c316e922eaf None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:23.667034 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7239ae-8e9b-4654-8945-0c316e922eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:23.667315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7239ae-8e9b-4654-8945-0c316e922eaf None admin] PATH_INFO: `/v3/projects/c4f989e5fad14bcebae5e279f22ee21d` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:23.668499 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7239ae-8e9b-4654-8945-0c316e922eaf None admin] RBAC: Authorizing `identity:get_project(project_id=c4f989e5fad14bcebae5e279f22ee21d)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:23.670029 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7239ae-8e9b-4654-8945-0c316e922eaf None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:23.670168 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68fd0d-d282-4995-9941-d9434acf4cc7 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:23.671986 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 100/195] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:23 2022] GET /identity/v3/projects/c4f989e5fad14bcebae5e279f22ee21d => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:23.672561 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7239ae-8e9b-4654-8945-0c316e922eaf None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:23.709736 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41867454-266b-412f-a8cc-40e201c84598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:23.710434 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41867454-266b-412f-a8cc-40e201c84598 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:23.710721 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41867454-266b-412f-a8cc-40e201c84598 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:23.710996 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41867454-266b-412f-a8cc-40e201c84598 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:23.711708 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41867454-266b-412f-a8cc-40e201c84598 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:23.713152 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41867454-266b-412f-a8cc-40e201c84598 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:23.722057 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:23.733486 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 96/196] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Jun 15 19:24:23 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:24:23.734311 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41867454-266b-412f-a8cc-40e201c84598 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:23.789174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7239ae-8e9b-4654-8945-0c316e922eaf None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:23.827141 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2768eb0-3aea-4f03-80e1-9b2c92c6cdf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:23.828161 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2768eb0-3aea-4f03-80e1-9b2c92c6cdf9 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:23.828520 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2768eb0-3aea-4f03-80e1-9b2c92c6cdf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:23.828795 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2768eb0-3aea-4f03-80e1-9b2c92c6cdf9 None admin] PATH_INFO: `/v3/users/1041c9fb868f414798a4a50b20e459af` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:23.830074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2768eb0-3aea-4f03-80e1-9b2c92c6cdf9 None admin] RBAC: Authorizing `identity:get_user(user_id=1041c9fb868f414798a4a50b20e459af)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:23.831569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2768eb0-3aea-4f03-80e1-9b2c92c6cdf9 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:23.833263 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 101/197] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:24:23 2022] GET /identity/v3/users/1041c9fb868f414798a4a50b20e459af => generated 283 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:23.834050 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2768eb0-3aea-4f03-80e1-9b2c92c6cdf9 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:23.836850 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:23.856032 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41867454-266b-412f-a8cc-40e201c84598 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:23.895112 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27988508-e73d-42c4-a877-b2a6e760b3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:23.896124 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27988508-e73d-42c4-a877-b2a6e760b3d0 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:23.897102 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27988508-e73d-42c4-a877-b2a6e760b3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:23.897102 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27988508-e73d-42c4-a877-b2a6e760b3d0 None admin] PATH_INFO: `/v3/users/a1ad396a5e26438288c0d36a4fe00a05` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:23.898076 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27988508-e73d-42c4-a877-b2a6e760b3d0 None admin] RBAC: Authorizing `identity:get_user(user_id=a1ad396a5e26438288c0d36a4fe00a05)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:23.899585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27988508-e73d-42c4-a877-b2a6e760b3d0 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:23.901545 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 97/198] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:24:23 2022] GET /identity/v3/users/a1ad396a5e26438288c0d36a4fe00a05 => generated 289 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:23.902348 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27988508-e73d-42c4-a877-b2a6e760b3d0 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:23.955599 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2768eb0-3aea-4f03-80e1-9b2c92c6cdf9 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:23.976322 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:23.999800 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bcaf81-5698-43b7-8c3c-c8cb54753664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:24.000458 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bcaf81-5698-43b7-8c3c-c8cb54753664 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:24.000741 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bcaf81-5698-43b7-8c3c-c8cb54753664 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:24.001015 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bcaf81-5698-43b7-8c3c-c8cb54753664 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:24.029371 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27988508-e73d-42c4-a877-b2a6e760b3d0 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:24.073719 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a32f0-ed1a-49d5-a5d3-b1d6e45853e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:24.074678 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a32f0-ed1a-49d5-a5d3-b1d6e45853e0 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:24.075596 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a32f0-ed1a-49d5-a5d3-b1d6e45853e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:24.076294 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a32f0-ed1a-49d5-a5d3-b1d6e45853e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:24.077451 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a32f0-ed1a-49d5-a5d3-b1d6e45853e0 None admin] RBAC: Adding query filter params (role.id=3920d5126e22480ab9a4eec9cf0fd4e7, scope.project.id=c4f989e5fad14bcebae5e279f22ee21d, user.id=abb9d50717324682b8fc92f386605834) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:24.077772 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a32f0-ed1a-49d5-a5d3-b1d6e45853e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:24.079701 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a32f0-ed1a-49d5-a5d3-b1d6e45853e0 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:24.083033 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bcaf81-5698-43b7-8c3c-c8cb54753664 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:24.084994 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bcaf81-5698-43b7-8c3c-c8cb54753664 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:24.086959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:24.087977 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 98/199] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:24:23 2022] GET /identity/v3/role_assignments?user.id=abb9d50717324682b8fc92f386605834&scope.project.id=c4f989e5fad14bcebae5e279f22ee21d&role.id=3920d5126e22480ab9a4eec9cf0fd4e7 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:24.089115 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a32f0-ed1a-49d5-a5d3-b1d6e45853e0 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:24.254235 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 102/200] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:24:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 19:24:24.256492 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bcaf81-5698-43b7-8c3c-c8cb54753664 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:24.267539 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a32f0-ed1a-49d5-a5d3-b1d6e45853e0 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:24.309774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:24.316138 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b79e69-dd87-4e19-89b3-3f5b87d308ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:24.317313 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b79e69-dd87-4e19-89b3-3f5b87d308ac None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:24.317609 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b79e69-dd87-4e19-89b3-3f5b87d308ac None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:24.318801 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b79e69-dd87-4e19-89b3-3f5b87d308ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:24.389488 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bcaf81-5698-43b7-8c3c-c8cb54753664 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:24.391049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:24.406770 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b79e69-dd87-4e19-89b3-3f5b87d308ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:24.408241 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b79e69-dd87-4e19-89b3-3f5b87d308ac None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:24.427450 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c341942-44f9-4aaa-aad6-e01923638643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:24.428108 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c341942-44f9-4aaa-aad6-e01923638643 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:24.428396 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c341942-44f9-4aaa-aad6-e01923638643 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:24.428675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c341942-44f9-4aaa-aad6-e01923638643 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:24.501602 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c341942-44f9-4aaa-aad6-e01923638643 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:24.503128 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c341942-44f9-4aaa-aad6-e01923638643 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:24.514434 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 99/201] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:24:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 19:24:24.515527 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b79e69-dd87-4e19-89b3-3f5b87d308ac None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:24.544271 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:24.615649 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 103/202] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:24:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 19:24:24.616445 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c341942-44f9-4aaa-aad6-e01923638643 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:24.618872 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:24.620811 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b79e69-dd87-4e19-89b3-3f5b87d308ac None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:24.656979 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7229f8-41fc-4819-9f56-6e535fe88b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:24.657846 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7229f8-41fc-4819-9f56-6e535fe88b83 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:24.658216 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7229f8-41fc-4819-9f56-6e535fe88b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:24.658662 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7229f8-41fc-4819-9f56-6e535fe88b83 None admin] PATH_INFO: `/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:24.659932 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7229f8-41fc-4819-9f56-6e535fe88b83 None admin] RBAC: Authorizing `identity:get_project(project_id=383a1a7fd67a4b0c8baf1403a8b02f25)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:24.661395 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7229f8-41fc-4819-9f56-6e535fe88b83 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:24.663441 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 100/203] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:24 2022] GET /identity/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:24.664223 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7229f8-41fc-4819-9f56-6e535fe88b83 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:24.736399 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c341942-44f9-4aaa-aad6-e01923638643 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:24.762611 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:24.769579 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7229f8-41fc-4819-9f56-6e535fe88b83 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:24.777213 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04f743-73fb-41ab-8b15-4414f479003a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:24.778751 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04f743-73fb-41ab-8b15-4414f479003a None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:24.779037 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04f743-73fb-41ab-8b15-4414f479003a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:24.779314 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04f743-73fb-41ab-8b15-4414f479003a None admin] PATH_INFO: `/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:24.780631 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d04f743-73fb-41ab-8b15-4414f479003a None admin] RBAC: Authorizing `identity:get_project(project_id=383a1a7fd67a4b0c8baf1403a8b02f25)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:24.782243 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d04f743-73fb-41ab-8b15-4414f479003a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:24.784326 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 104/204] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:24 2022] GET /identity/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:24.785221 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04f743-73fb-41ab-8b15-4414f479003a None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:24.806124 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374387fa-346c-495e-a3cc-43f824680f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:24.806991 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374387fa-346c-495e-a3cc-43f824680f90 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:24.807270 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374387fa-346c-495e-a3cc-43f824680f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:24.807543 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374387fa-346c-495e-a3cc-43f824680f90 None admin] PATH_INFO: `/v3/projects/c4f989e5fad14bcebae5e279f22ee21d` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:24.808729 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374387fa-346c-495e-a3cc-43f824680f90 None admin] RBAC: Authorizing `identity:get_project(project_id=c4f989e5fad14bcebae5e279f22ee21d)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:24.810244 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374387fa-346c-495e-a3cc-43f824680f90 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:24.812242 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 101/205] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:24 2022] GET /identity/v3/projects/c4f989e5fad14bcebae5e279f22ee21d => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:24.813190 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374387fa-346c-495e-a3cc-43f824680f90 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:24.860707 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:24.895963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04f743-73fb-41ab-8b15-4414f479003a None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:24.914982 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374387fa-346c-495e-a3cc-43f824680f90 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:24.932635 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5546ae4-5a8d-46ec-9a28-fdc734a771d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:24.933296 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5546ae4-5a8d-46ec-9a28-fdc734a771d9 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:24.933582 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5546ae4-5a8d-46ec-9a28-fdc734a771d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:24.933861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5546ae4-5a8d-46ec-9a28-fdc734a771d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:24.934761 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5546ae4-5a8d-46ec-9a28-fdc734a771d9 None admin] RBAC: Adding query filter params (role.id=4f736313c26d4f8db48bbb2f346e50e4, scope.project.id=383a1a7fd67a4b0c8baf1403a8b02f25, user.id=1041c9fb868f414798a4a50b20e459af) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:24.935183 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5546ae4-5a8d-46ec-9a28-fdc734a771d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:24.936628 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5546ae4-5a8d-46ec-9a28-fdc734a771d9 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:24.942900 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 105/206] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:24:24 2022] GET /identity/v3/role_assignments?user.id=1041c9fb868f414798a4a50b20e459af&scope.project.id=383a1a7fd67a4b0c8baf1403a8b02f25&role.id=4f736313c26d4f8db48bbb2f346e50e4 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:24.948986 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca3ced-1654-425b-850e-146e4227c7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:24.949661 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca3ced-1654-425b-850e-146e4227c7c1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:24.950507 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5546ae4-5a8d-46ec-9a28-fdc734a771d9 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:24.950620 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca3ced-1654-425b-850e-146e4227c7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:24.951078 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca3ced-1654-425b-850e-146e4227c7c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:24.952047 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca3ced-1654-425b-850e-146e4227c7c1 None admin] RBAC: Adding query filter params (role.id=3920d5126e22480ab9a4eec9cf0fd4e7, scope.project.id=383a1a7fd67a4b0c8baf1403a8b02f25, user.id=1041c9fb868f414798a4a50b20e459af) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:24.952452 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca3ced-1654-425b-850e-146e4227c7c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:24.954057 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca3ced-1654-425b-850e-146e4227c7c1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:24.960462 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 102/207] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:24:24 2022] GET /identity/v3/role_assignments?user.id=1041c9fb868f414798a4a50b20e459af&scope.project.id=383a1a7fd67a4b0c8baf1403a8b02f25&role.id=3920d5126e22480ab9a4eec9cf0fd4e7 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:25.123883 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5546ae4-5a8d-46ec-9a28-fdc734a771d9 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:25.170617 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7782634-6c9d-40a6-8d47-3b88cf15139c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:25.171221 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7782634-6c9d-40a6-8d47-3b88cf15139c None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:25.171502 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7782634-6c9d-40a6-8d47-3b88cf15139c None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:25.171779 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7782634-6c9d-40a6-8d47-3b88cf15139c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:25.172657 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7782634-6c9d-40a6-8d47-3b88cf15139c None admin] RBAC: Adding query filter params (role.id=4e60cc1f881a4caebdc208088b7ffcaa, scope.project.id=c4f989e5fad14bcebae5e279f22ee21d, user.id=a1ad396a5e26438288c0d36a4fe00a05) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:25.172973 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7782634-6c9d-40a6-8d47-3b88cf15139c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:25.174526 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7782634-6c9d-40a6-8d47-3b88cf15139c None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:25.177258 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:25.185485 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 106/208] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:24:24 2022] GET /identity/v3/role_assignments?user.id=a1ad396a5e26438288c0d36a4fe00a05&scope.project.id=c4f989e5fad14bcebae5e279f22ee21d&role.id=4e60cc1f881a4caebdc208088b7ffcaa => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:26.199890 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf735110-4264-4a06-9d6e-5e3a77852c17 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:26.200407 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf735110-4264-4a06-9d6e-5e3a77852c17 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:26.200482 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf735110-4264-4a06-9d6e-5e3a77852c17 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:26.201977 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 103/209] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:26.207889 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e0d46-45b6-4f6c-a353-c064ff148a39 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:26.208865 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e0d46-45b6-4f6c-a353-c064ff148a39 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:26.209357 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e0d46-45b6-4f6c-a353-c064ff148a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:26.227893 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-c25e0d46-45b6-4f6c-a353-c064ff148a39 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:26.229399 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-c25e0d46-45b6-4f6c-a353-c064ff148a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:26.308958 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 107/210] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 19:24:26.332147 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf735110-4264-4a06-9d6e-5e3a77852c17 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:26.346493 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9c333-e250-4b65-ac01-9af770cc0c59 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:26.346780 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9c333-e250-4b65-ac01-9af770cc0c59 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:26.347055 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9c333-e250-4b65-ac01-9af770cc0c59 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:26.348717 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 108/211] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 19:24:26.355361 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281d7c8-8c90-4237-9ebd-f99eca2c0d08 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:26.355644 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281d7c8-8c90-4237-9ebd-f99eca2c0d08 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:26.355916 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281d7c8-8c90-4237-9ebd-f99eca2c0d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:26.363509 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:26.375000 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-3281d7c8-8c90-4237-9ebd-f99eca2c0d08 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:26.376177 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-3281d7c8-8c90-4237-9ebd-f99eca2c0d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:26.444088 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf735110-4264-4a06-9d6e-5e3a77852c17 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:26.452350 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 109/212] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:26.472543 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281d7c8-8c90-4237-9ebd-f99eca2c0d08 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:26.479832 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb78cd1-1198-4b42-8fa1-e981b01d6221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:26.480680 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb78cd1-1198-4b42-8fa1-e981b01d6221 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:26.481485 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb78cd1-1198-4b42-8fa1-e981b01d6221 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:26.481805 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb78cd1-1198-4b42-8fa1-e981b01d6221 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:26.492438 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb78cd1-1198-4b42-8fa1-e981b01d6221 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:26.500303 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-6cb78cd1-1198-4b42-8fa1-e981b01d6221 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 19:24:26.500303 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:26.500303 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:26.500303 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:26.500303 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:26.500303 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:26.500303 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:26.500303 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:26.500303 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:26.500303 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:26.500303 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:26.500303 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:26.500303 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:26.500303 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:26.500303 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:26.500303 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:26.501189 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:26.501189 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:26.501189 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:26.501189 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:26.501189 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:26.501189 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:26.501189 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:26.501189 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:26.501189 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:26.501189 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:26.501189 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:26.501189 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:26.501189 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:26.501189 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:26.501189 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:26.501189 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:26.501988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:26.501988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:26.501988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:26.501988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:26.501988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:26.501988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:26.501988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:26.501988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:26.501988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:26.501988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:26.501988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 19:24:26.501988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:24:26.501988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 104/213] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:24:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:26.507079 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb78cd1-1198-4b42-8fa1-e981b01d6221 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:26.542650 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:26.580541 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281d7c8-8c90-4237-9ebd-f99eca2c0d08 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:26.595673 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:26.618119 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb78cd1-1198-4b42-8fa1-e981b01d6221 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:26.618711 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61969722-e1bc-413a-a99d-3842f4e03709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:26.619509 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61969722-e1bc-413a-a99d-3842f4e03709 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:26.619889 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61969722-e1bc-413a-a99d-3842f4e03709 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:26.620164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61969722-e1bc-413a-a99d-3842f4e03709 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:26.621391 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61969722-e1bc-413a-a99d-3842f4e03709 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:26.623647 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61969722-e1bc-413a-a99d-3842f4e03709 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:26.625325 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 110/214] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:24:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:26.630774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61969722-e1bc-413a-a99d-3842f4e03709 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:26.659824 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f61bf-8752-4fee-90ec-52078a749d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:26.660498 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f61bf-8752-4fee-90ec-52078a749d39 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:26.660932 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f61bf-8752-4fee-90ec-52078a749d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:26.661217 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f61bf-8752-4fee-90ec-52078a749d39 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:26.662332 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f61bf-8752-4fee-90ec-52078a749d39 None admin] RBAC: Adding query filter params (name=member) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:26.662909 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f61bf-8752-4fee-90ec-52078a749d39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:26.664461 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f61bf-8752-4fee-90ec-52078a749d39 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:26.678334 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 105/215] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jun 15 19:24:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:24:26.683735 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f61bf-8752-4fee-90ec-52078a749d39 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:26.744224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61969722-e1bc-413a-a99d-3842f4e03709 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:26.772031 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:26.781837 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac3a29e-af75-49e6-9045-0d58b261f7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:26.782611 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3a29e-af75-49e6-9045-0d58b261f7c9 None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:26.782957 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3a29e-af75-49e6-9045-0d58b261f7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:26.783301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3a29e-af75-49e6-9045-0d58b261f7c9 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:26.784078 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac3a29e-af75-49e6-9045-0d58b261f7c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:26.785628 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac3a29e-af75-49e6-9045-0d58b261f7c9 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:26.795168 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f61bf-8752-4fee-90ec-52078a749d39 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:26.810253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 111/216] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Jun 15 19:24:26 2022] POST /identity/v3/users => generated 297 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:24:26.835311 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dda08d-4f26-4c9c-bf2e-68f8df35df4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:26.836221 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dda08d-4f26-4c9c-bf2e-68f8df35df4f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:26.836502 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dda08d-4f26-4c9c-bf2e-68f8df35df4f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:26.836781 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dda08d-4f26-4c9c-bf2e-68f8df35df4f None admin] PATH_INFO: `/v3/users/e610cca9a3ba43a29f4f207fd2e1eb45` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:26.853629 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dda08d-4f26-4c9c-bf2e-68f8df35df4f None admin] RBAC: Authorizing `identity:get_user(user_id=e610cca9a3ba43a29f4f207fd2e1eb45)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:26.855213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dda08d-4f26-4c9c-bf2e-68f8df35df4f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:26.857557 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 106/217] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:24:26 2022] GET /identity/v3/users/e610cca9a3ba43a29f4f207fd2e1eb45 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:24:26.863211 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac3a29e-af75-49e6-9045-0d58b261f7c9 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:26.980381 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac3a29e-af75-49e6-9045-0d58b261f7c9 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:27.018373 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429f242a-9845-4fd5-b068-11b5449e145a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:27.018959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f242a-9845-4fd5-b068-11b5449e145a None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:27.019239 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f242a-9845-4fd5-b068-11b5449e145a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:27.019515 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f242a-9845-4fd5-b068-11b5449e145a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:27.072723 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:27.095986 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429f242a-9845-4fd5-b068-11b5449e145a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:27.097472 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429f242a-9845-4fd5-b068-11b5449e145a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:27.205443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 112/218] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:24:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 19:24:27.214225 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dda08d-4f26-4c9c-bf2e-68f8df35df4f None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:27.231080 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:27.298008 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b1e23-c098-4ec9-9f47-59cf4e47be82 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:27.298514 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b1e23-c098-4ec9-9f47-59cf4e47be82 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:27.298514 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b1e23-c098-4ec9-9f47-59cf4e47be82 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:27.300667 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 113/219] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:27.306412 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09d4f3-9fb9-4166-99d6-e00012d34956 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:27.306706 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09d4f3-9fb9-4166-99d6-e00012d34956 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:27.306987 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09d4f3-9fb9-4166-99d6-e00012d34956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:27.330991 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-8c09d4f3-9fb9-4166-99d6-e00012d34956 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:27.332225 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-8c09d4f3-9fb9-4166-99d6-e00012d34956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:27.356953 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dda08d-4f26-4c9c-bf2e-68f8df35df4f None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:27.402380 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a39cbb-c417-4c1c-a58c-26e2e9d417c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:27.403825 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a39cbb-c417-4c1c-a58c-26e2e9d417c5 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:27.404166 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a39cbb-c417-4c1c-a58c-26e2e9d417c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:27.404438 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a39cbb-c417-4c1c-a58c-26e2e9d417c5 None admin] PATH_INFO: `/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:27.405669 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a39cbb-c417-4c1c-a58c-26e2e9d417c5 None admin] RBAC: Authorizing `identity:get_project(project_id=383a1a7fd67a4b0c8baf1403a8b02f25)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:27.408104 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a39cbb-c417-4c1c-a58c-26e2e9d417c5 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:27.409831 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 107/220] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:27 2022] GET /identity/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:24:27.415698 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a39cbb-c417-4c1c-a58c-26e2e9d417c5 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:27.442880 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 114/221] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:27.444750 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0317c10-5b59-4068-b7a8-ff5f40ddfd2a None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:27.445038 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0317c10-5b59-4068-b7a8-ff5f40ddfd2a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:27.445315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0317c10-5b59-4068-b7a8-ff5f40ddfd2a None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:27.446820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 115/222] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:27.460274 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93a8cf-bf77-4170-ba1a-ef8b961bbaaf None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:27.460570 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93a8cf-bf77-4170-ba1a-ef8b961bbaaf None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:27.460830 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93a8cf-bf77-4170-ba1a-ef8b961bbaaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:27.487733 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-5e93a8cf-bf77-4170-ba1a-ef8b961bbaaf None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:27.489652 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-5e93a8cf-bf77-4170-ba1a-ef8b961bbaaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:27.501058 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:27.521970 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:27.551659 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a39cbb-c417-4c1c-a58c-26e2e9d417c5 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:27.585604 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 116/223] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 15 19:24:27.586601 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e93a8cf-bf77-4170-ba1a-ef8b961bbaaf None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:27.607251 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0436ec6a-0756-4590-b6c0-85bba0bccc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:27.607979 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0436ec6a-0756-4590-b6c0-85bba0bccc65 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:27.608266 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0436ec6a-0756-4590-b6c0-85bba0bccc65 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:27.608596 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0436ec6a-0756-4590-b6c0-85bba0bccc65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:27.609454 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0436ec6a-0756-4590-b6c0-85bba0bccc65 None admin] RBAC: Adding query filter params (role.id=e53a78912e764f8387e0cf637bb86bee, scope.project.id=383a1a7fd67a4b0c8baf1403a8b02f25, user.id=e610cca9a3ba43a29f4f207fd2e1eb45) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:27.609773 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0436ec6a-0756-4590-b6c0-85bba0bccc65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:27.611331 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0436ec6a-0756-4590-b6c0-85bba0bccc65 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:27.623248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 108/224] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:24:27 2022] GET /identity/v3/role_assignments?user.id=e610cca9a3ba43a29f4f207fd2e1eb45&scope.project.id=383a1a7fd67a4b0c8baf1403a8b02f25&role.id=e53a78912e764f8387e0cf637bb86bee => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:24:27.639243 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0436ec6a-0756-4590-b6c0-85bba0bccc65 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:27.714614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e93a8cf-bf77-4170-ba1a-ef8b961bbaaf None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:27.757377 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c24e2cd-9d43-4360-a8d6-ae617dba191e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:27.758762 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24e2cd-9d43-4360-a8d6-ae617dba191e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:27.759314 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24e2cd-9d43-4360-a8d6-ae617dba191e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:27.759592 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24e2cd-9d43-4360-a8d6-ae617dba191e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:27.772365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c24e2cd-9d43-4360-a8d6-ae617dba191e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:27.782545 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-4c24e2cd-9d43-4360-a8d6-ae617dba191e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 19:24:27.782545 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:27.782545 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:27.782545 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:27.782545 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:27.782545 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:27.782545 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:27.782545 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:27.782545 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:27.782545 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:27.782545 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:27.782545 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:27.782545 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:27.782545 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:27.782545 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:27.782545 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:27.783483 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:27.783483 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:27.783483 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:27.783483 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:27.783483 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:27.783483 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:27.783483 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:27.783483 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:27.783483 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:27.783483 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:27.783483 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:27.783483 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:27.783483 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:27.783483 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:27.783483 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:27.783483 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:27.784301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:27.784301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:27.784301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:27.784301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:27.784301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:27.784301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:27.784301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:27.784301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:27.784301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:27.784301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:27.784301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 19:24:27.784301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:24:27.784907 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 117/225] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jun 15 19:24:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:27.790898 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c24e2cd-9d43-4360-a8d6-ae617dba191e None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:27.794231 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:27.874002 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0436ec6a-0756-4590-b6c0-85bba0bccc65 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:27.913296 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a7713c-eab6-4a29-a27a-fd28d2412c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:27.914126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7713c-eab6-4a29-a27a-fd28d2412c61 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:27.914410 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7713c-eab6-4a29-a27a-fd28d2412c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:27.914694 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7713c-eab6-4a29-a27a-fd28d2412c61 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:27.920126 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c24e2cd-9d43-4360-a8d6-ae617dba191e None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:27.924007 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:27.929075 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a7713c-eab6-4a29-a27a-fd28d2412c61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:27.937798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-e0a7713c-eab6-4a29-a27a-fd28d2412c61 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:24:27.937798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:27.937798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:27.937798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:27.937798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:27.937798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:27.937798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:27.937798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:27.937798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:27.937798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:27.937798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:27.937798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:27.937798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:27.937798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:27.937798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:27.937798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:27.939581 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:27.939581 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:27.939581 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:27.939581 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:27.939581 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:27.939581 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:27.939581 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:27.939581 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:27.939581 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:27.939581 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:27.939581 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:27.939581 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:27.939581 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:27.939581 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:27.939581 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:27.939581 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:27.941174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:27.941174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:27.941174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:27.941174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:27.941174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:27.941174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:27.941174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:27.941174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:27.941174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:27.941174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:27.941174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:24:27.941174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:24:27.941174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 109/226] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jun 15 19:24:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:27.946556 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a7713c-eab6-4a29-a27a-fd28d2412c61 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:27.960188 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415bd360-b314-4478-8459-f0841eb57395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:27.960872 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415bd360-b314-4478-8459-f0841eb57395 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:27.961474 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415bd360-b314-4478-8459-f0841eb57395 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:27.961756 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415bd360-b314-4478-8459-f0841eb57395 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:27.962837 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415bd360-b314-4478-8459-f0841eb57395 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:27.963152 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415bd360-b314-4478-8459-f0841eb57395 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:27.965050 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415bd360-b314-4478-8459-f0841eb57395 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:27.975427 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 118/227] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:24:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:27.981603 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415bd360-b314-4478-8459-f0841eb57395 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:28.051383 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:28.093861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415bd360-b314-4478-8459-f0841eb57395 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:28.130350 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69250d0-6327-4f61-b4de-72e0a9ba9029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:28.131445 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69250d0-6327-4f61-b4de-72e0a9ba9029 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:28.131732 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69250d0-6327-4f61-b4de-72e0a9ba9029 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:28.132011 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69250d0-6327-4f61-b4de-72e0a9ba9029 None admin] PATH_INFO: `/v3/users/1041c9fb868f414798a4a50b20e459af` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:28.133222 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69250d0-6327-4f61-b4de-72e0a9ba9029 None admin] RBAC: Authorizing `identity:get_user(user_id=1041c9fb868f414798a4a50b20e459af)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:28.135027 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69250d0-6327-4f61-b4de-72e0a9ba9029 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:28.142296 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 119/228] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:24:27 2022] GET /identity/v3/users/1041c9fb868f414798a4a50b20e459af => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:24:28.143509 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69250d0-6327-4f61-b4de-72e0a9ba9029 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:28.179977 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a7713c-eab6-4a29-a27a-fd28d2412c61 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:28.251311 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69250d0-6327-4f61-b4de-72e0a9ba9029 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:28.283023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b2f8a-a947-4722-94f8-783be25559cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:28.283708 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b2f8a-a947-4722-94f8-783be25559cc None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:28.283997 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b2f8a-a947-4722-94f8-783be25559cc None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:28.284391 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b2f8a-a947-4722-94f8-783be25559cc None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:28.284727 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:28.285165 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239b2f8a-a947-4722-94f8-783be25559cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:28.285574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239b2f8a-a947-4722-94f8-783be25559cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:28.287280 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239b2f8a-a947-4722-94f8-783be25559cc None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:28.288086 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38b1f6-9506-4558-b6f9-d4dbbb7ee64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:28.289030 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38b1f6-9506-4558-b6f9-d4dbbb7ee64e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:28.289330 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38b1f6-9506-4558-b6f9-d4dbbb7ee64e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:28.289628 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38b1f6-9506-4558-b6f9-d4dbbb7ee64e None admin] PATH_INFO: `/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:28.298993 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c38b1f6-9506-4558-b6f9-d4dbbb7ee64e None admin] RBAC: Authorizing `identity:get_project(project_id=383a1a7fd67a4b0c8baf1403a8b02f25)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:28.300565 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c38b1f6-9506-4558-b6f9-d4dbbb7ee64e None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:28.310322 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 120/229] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:28 2022] GET /identity/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:28.312147 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38b1f6-9506-4558-b6f9-d4dbbb7ee64e None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:28.312974 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 110/230] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:24:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:28.319001 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b2f8a-a947-4722-94f8-783be25559cc None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:28.416404 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:28.465927 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b2f8a-a947-4722-94f8-783be25559cc None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:28.508685 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe9571-36ed-4bce-86cb-0596949190ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:28.510217 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe9571-36ed-4bce-86cb-0596949190ed None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:28.511923 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe9571-36ed-4bce-86cb-0596949190ed None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:28.513780 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe9571-36ed-4bce-86cb-0596949190ed None admin] PATH_INFO: `/v3/users/1041c9fb868f414798a4a50b20e459af` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:28.516291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fe9571-36ed-4bce-86cb-0596949190ed None admin] RBAC: Authorizing `identity:get_user(user_id=1041c9fb868f414798a4a50b20e459af)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:28.517946 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fe9571-36ed-4bce-86cb-0596949190ed None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:28.519831 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 111/231] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:24:28 2022] GET /identity/v3/users/1041c9fb868f414798a4a50b20e459af => generated 283 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:28.525364 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe9571-36ed-4bce-86cb-0596949190ed None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:28.553721 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38b1f6-9506-4558-b6f9-d4dbbb7ee64e None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:28.646010 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe9571-36ed-4bce-86cb-0596949190ed None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:28.670993 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57699119-69ab-4721-8b17-95db9d66e537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:28.671670 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57699119-69ab-4721-8b17-95db9d66e537 None admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:28.671952 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57699119-69ab-4721-8b17-95db9d66e537 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:28.672224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57699119-69ab-4721-8b17-95db9d66e537 None admin] PATH_INFO: `/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25/users/1041c9fb868f414798a4a50b20e459af/roles/3920d5126e22480ab9a4eec9cf0fd4e7` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:28.674823 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:28.683308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57699119-69ab-4721-8b17-95db9d66e537 None admin] RBAC: Authorizing `identity:create_grant(project_id=383a1a7fd67a4b0c8baf1403a8b02f25, user_id=1041c9fb868f414798a4a50b20e459af, role_id=3920d5126e22480ab9a4eec9cf0fd4e7)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:28.685494 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57699119-69ab-4721-8b17-95db9d66e537 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:28.690377 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3024d35-5484-49d1-aa76-b7cd24f60c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:28.691466 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3024d35-5484-49d1-aa76-b7cd24f60c03 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:28.691969 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3024d35-5484-49d1-aa76-b7cd24f60c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:28.692435 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3024d35-5484-49d1-aa76-b7cd24f60c03 None admin] PATH_INFO: `/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:28.694239 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3024d35-5484-49d1-aa76-b7cd24f60c03 None admin] RBAC: Authorizing `identity:get_project(project_id=383a1a7fd67a4b0c8baf1403a8b02f25)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:28.695996 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3024d35-5484-49d1-aa76-b7cd24f60c03 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:28.699016 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 112/232] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:28 2022] GET /identity/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:28.705066 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3024d35-5484-49d1-aa76-b7cd24f60c03 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:28.722541 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 121/233] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jun 15 19:24:28 2022] PUT /identity/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25/users/1041c9fb868f414798a4a50b20e459af/roles/3920d5126e22480ab9a4eec9cf0fd4e7 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:24:28.824048 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3024d35-5484-49d1-aa76-b7cd24f60c03 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:28.864045 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cfdf56-b005-426e-a317-c63936d5a2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:28.865016 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cfdf56-b005-426e-a317-c63936d5a2c0 None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:28.865302 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cfdf56-b005-426e-a317-c63936d5a2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:28.865580 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cfdf56-b005-426e-a317-c63936d5a2c0 None admin] PATH_INFO: `/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25/users/1041c9fb868f414798a4a50b20e459af/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:28.867754 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cfdf56-b005-426e-a317-c63936d5a2c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=383a1a7fd67a4b0c8baf1403a8b02f25, user_id=1041c9fb868f414798a4a50b20e459af, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:28.869241 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cfdf56-b005-426e-a317-c63936d5a2c0 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:28.893786 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 113/234] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jun 15 19:24:28 2022] PUT /identity/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25/users/1041c9fb868f414798a4a50b20e459af/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:24:29.538874 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b710e9b-ed51-4806-b1c1-83f5f840c6b2 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:29.539873 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b710e9b-ed51-4806-b1c1-83f5f840c6b2 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:29.540375 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b710e9b-ed51-4806-b1c1-83f5f840c6b2 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:29.541951 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 122/235] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:29.547829 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e835178-adec-492b-878f-a655682bfe65 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:29.548238 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e835178-adec-492b-878f-a655682bfe65 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:29.548408 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e835178-adec-492b-878f-a655682bfe65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:29.579800 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-3e835178-adec-492b-878f-a655682bfe65 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:29.580999 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3e835178-adec-492b-878f-a655682bfe65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:29.653154 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:29.678357 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 114/236] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 19:24:29.702081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b710e9b-ed51-4806-b1c1-83f5f840c6b2 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:29.770890 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:29.830398 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b710e9b-ed51-4806-b1c1-83f5f840c6b2 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:29.865204 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee0c08-1aa2-4ab3-a3e6-57c0c385df27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:29.866000 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee0c08-1aa2-4ab3-a3e6-57c0c385df27 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:29.866318 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee0c08-1aa2-4ab3-a3e6-57c0c385df27 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:29.866594 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee0c08-1aa2-4ab3-a3e6-57c0c385df27 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:29.877670 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ee0c08-1aa2-4ab3-a3e6-57c0c385df27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:29.885892 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-04ee0c08-1aa2-4ab3-a3e6-57c0c385df27 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 19:24:29.885892 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:29.885892 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:29.885892 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:29.885892 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:29.885892 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:29.885892 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:29.885892 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:29.885892 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:29.885892 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:29.885892 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:29.885892 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:29.885892 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:29.885892 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:29.885892 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:29.885892 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:29.887001 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:29.887001 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:29.887001 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:29.887001 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:29.887001 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:29.887001 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:29.887001 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:29.887001 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:29.887001 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:29.887001 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:29.887001 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:29.887001 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:29.887001 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:29.887001 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:29.887001 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:29.887001 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:29.888141 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:29.888141 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:29.888141 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:29.888141 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:29.888141 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:29.888141 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:29.888141 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:29.888141 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:29.888141 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:29.888141 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:29.888141 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 19:24:29.888141 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:24:29.888141 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 123/237] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:24:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:29.893415 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e835178-adec-492b-878f-a655682bfe65 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:29.910259 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06dadb0-50df-4c8a-9527-f11bafbb13c8 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:29.910473 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06dadb0-50df-4c8a-9527-f11bafbb13c8 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:29.910755 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06dadb0-50df-4c8a-9527-f11bafbb13c8 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:29.912424 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 124/238] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 19:24:29.917276 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12906f25-79b5-4936-8c75-f087db4d35fa None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:29.917568 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12906f25-79b5-4936-8c75-f087db4d35fa None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:29.917849 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12906f25-79b5-4936-8c75-f087db4d35fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:29.954447 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-12906f25-79b5-4936-8c75-f087db4d35fa None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:29.955673 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-12906f25-79b5-4936-8c75-f087db4d35fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:30.075258 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 125/239] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 19:24:30.076436 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b61ae-d647-434f-b7ce-f9295a8956b0 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:30.076731 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b61ae-d647-434f-b7ce-f9295a8956b0 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:30.077009 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b61ae-d647-434f-b7ce-f9295a8956b0 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:30.078386 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 126/240] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:30.082237 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e835178-adec-492b-878f-a655682bfe65 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:30.083712 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb7995-296b-43bb-9c5a-f2171a67c182 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:30.083998 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb7995-296b-43bb-9c5a-f2171a67c182 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:30.084275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb7995-296b-43bb-9c5a-f2171a67c182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:30.101367 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-e0bb7995-296b-43bb-9c5a-f2171a67c182 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:30.102661 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-e0bb7995-296b-43bb-9c5a-f2171a67c182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:30.117064 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640236c4-adee-4d06-85f4-8ee71b3c7e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:30.118047 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640236c4-adee-4d06-85f4-8ee71b3c7e78 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:30.118357 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640236c4-adee-4d06-85f4-8ee71b3c7e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:30.118630 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640236c4-adee-4d06-85f4-8ee71b3c7e78 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:30.119391 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640236c4-adee-4d06-85f4-8ee71b3c7e78 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:30.119694 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640236c4-adee-4d06-85f4-8ee71b3c7e78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:30.121066 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640236c4-adee-4d06-85f4-8ee71b3c7e78 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:30.130457 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 115/241] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jun 15 19:24:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:30.131269 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640236c4-adee-4d06-85f4-8ee71b3c7e78 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:30.158611 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:30.183510 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 127/242] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 19:24:30.184365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb7995-296b-43bb-9c5a-f2171a67c182 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:30.202545 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:30.240742 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640236c4-adee-4d06-85f4-8ee71b3c7e78 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:30.275625 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2c0e2-fc4f-4393-aa32-7ba08a20dfa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:30.276469 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2c0e2-fc4f-4393-aa32-7ba08a20dfa9 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:30.276753 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2c0e2-fc4f-4393-aa32-7ba08a20dfa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:30.277024 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2c0e2-fc4f-4393-aa32-7ba08a20dfa9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:30.278797 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d2c0e2-fc4f-4393-aa32-7ba08a20dfa9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:30.280243 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d2c0e2-fc4f-4393-aa32-7ba08a20dfa9 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:30.282269 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 116/243] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:24:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:30.283447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2c0e2-fc4f-4393-aa32-7ba08a20dfa9 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:30.313463 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb7995-296b-43bb-9c5a-f2171a67c182 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:30.348800 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d557bd-41d4-4ec8-9738-5c82005da784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:30.349687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d557bd-41d4-4ec8-9738-5c82005da784 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:30.350188 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d557bd-41d4-4ec8-9738-5c82005da784 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:30.350653 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d557bd-41d4-4ec8-9738-5c82005da784 None admin] PATH_INFO: `/v3/users/a574aa2e7fa843dda8e77b065bb25c71` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:30.365046 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d557bd-41d4-4ec8-9738-5c82005da784 None admin] RBAC: Authorizing `identity:get_user(user_id=a574aa2e7fa843dda8e77b065bb25c71)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:30.366884 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d557bd-41d4-4ec8-9738-5c82005da784 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:30.368810 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 128/244] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:24:30 2022] GET /identity/v3/users/a574aa2e7fa843dda8e77b065bb25c71 => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:30.374265 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d557bd-41d4-4ec8-9738-5c82005da784 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:30.390514 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2c0e2-fc4f-4393-aa32-7ba08a20dfa9 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:30.415494 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:30.418891 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:30.429896 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e153aa-895f-4c60-948d-391005935661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:30.431073 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e153aa-895f-4c60-948d-391005935661 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:30.431557 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e153aa-895f-4c60-948d-391005935661 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:30.431971 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e153aa-895f-4c60-948d-391005935661 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:30.444806 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e153aa-895f-4c60-948d-391005935661 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:30.453852 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-b5e153aa-895f-4c60-948d-391005935661 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 19:24:30.453852 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:30.453852 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:30.453852 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:30.453852 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:30.453852 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:30.453852 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:30.453852 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:30.453852 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:30.453852 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:30.453852 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:30.453852 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:30.453852 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:30.453852 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:30.453852 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:30.453852 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:30.454956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:30.454956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:30.454956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:30.454956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:30.454956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:30.454956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:30.454956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:30.454956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:30.454956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:30.454956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:30.454956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:30.454956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:30.454956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:30.454956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:30.454956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:30.454956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:30.455894 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:30.455894 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:30.455894 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:30.455894 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:30.455894 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:30.455894 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:30.455894 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:30.455894 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:30.455894 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:30.455894 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:30.455894 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 19:24:30.455894 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:24:30.458315 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 117/245] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:24:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:30.459219 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e153aa-895f-4c60-948d-391005935661 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:30.487922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d557bd-41d4-4ec8-9738-5c82005da784 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:30.525746 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c8a10-d693-4e1f-94af-ad1dbc15ec1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:30.526554 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c8a10-d693-4e1f-94af-ad1dbc15ec1d None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:30.526848 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c8a10-d693-4e1f-94af-ad1dbc15ec1d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:30.527130 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c8a10-d693-4e1f-94af-ad1dbc15ec1d None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:30.528386 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c8a10-d693-4e1f-94af-ad1dbc15ec1d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:30.529927 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c8a10-d693-4e1f-94af-ad1dbc15ec1d None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:30.554074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 129/246] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Jun 15 19:24:30 2022] POST /identity/v3/users => generated 293 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:24:30.555082 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c8a10-d693-4e1f-94af-ad1dbc15ec1d None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:30.573336 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e153aa-895f-4c60-948d-391005935661 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:30.614163 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814cc3c-e772-442c-baa3-b1ab87a3ac2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:30.615330 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814cc3c-e772-442c-baa3-b1ab87a3ac2b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:30.615831 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814cc3c-e772-442c-baa3-b1ab87a3ac2b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:30.616351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814cc3c-e772-442c-baa3-b1ab87a3ac2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:30.667189 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:30.669563 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:30.675893 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c8a10-d693-4e1f-94af-ad1dbc15ec1d None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:30.696523 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f814cc3c-e772-442c-baa3-b1ab87a3ac2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:30.698470 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f814cc3c-e772-442c-baa3-b1ab87a3ac2b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:30.716478 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3da5c6-64ea-4489-8a56-7dd26714eeb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:30.717177 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3da5c6-64ea-4489-8a56-7dd26714eeb6 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:30.717466 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3da5c6-64ea-4489-8a56-7dd26714eeb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:30.717747 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3da5c6-64ea-4489-8a56-7dd26714eeb6 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:30.725635 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3da5c6-64ea-4489-8a56-7dd26714eeb6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:30.725635 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3da5c6-64ea-4489-8a56-7dd26714eeb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:30.725635 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3da5c6-64ea-4489-8a56-7dd26714eeb6 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:30.731006 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 130/247] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jun 15 19:24:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:30.736964 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3da5c6-64ea-4489-8a56-7dd26714eeb6 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:30.860441 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 118/248] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:24:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 19:24:30.866289 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814cc3c-e772-442c-baa3-b1ab87a3ac2b None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:30.901460 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3da5c6-64ea-4489-8a56-7dd26714eeb6 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:30.945358 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd3b79-d88d-4880-be56-75bafc6a239e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:30.946881 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd3b79-d88d-4880-be56-75bafc6a239e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:30.947363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd3b79-d88d-4880-be56-75bafc6a239e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:30.947895 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd3b79-d88d-4880-be56-75bafc6a239e None admin] PATH_INFO: `/v3/users/e610cca9a3ba43a29f4f207fd2e1eb45` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:30.949338 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cd3b79-d88d-4880-be56-75bafc6a239e None admin] RBAC: Authorizing `identity:get_user(user_id=e610cca9a3ba43a29f4f207fd2e1eb45)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:30.951657 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cd3b79-d88d-4880-be56-75bafc6a239e None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:30.953833 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 131/249] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:24:30 2022] GET /identity/v3/users/e610cca9a3ba43a29f4f207fd2e1eb45 => generated 297 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:24:30.963638 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd3b79-d88d-4880-be56-75bafc6a239e None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:30.974983 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:31.004588 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814cc3c-e772-442c-baa3-b1ab87a3ac2b None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:31.005736 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:31.053341 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0acc8bd-c6e6-48af-9106-2fbdbe0c69e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:31.054243 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0acc8bd-c6e6-48af-9106-2fbdbe0c69e0 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:31.054531 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0acc8bd-c6e6-48af-9106-2fbdbe0c69e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:31.054809 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0acc8bd-c6e6-48af-9106-2fbdbe0c69e0 None admin] PATH_INFO: `/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:31.058542 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0acc8bd-c6e6-48af-9106-2fbdbe0c69e0 None admin] RBAC: Authorizing `identity:get_project(project_id=383a1a7fd67a4b0c8baf1403a8b02f25)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:31.060163 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0acc8bd-c6e6-48af-9106-2fbdbe0c69e0 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:31.062153 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 119/250] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:30 2022] GET /identity/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:24:31.069206 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0acc8bd-c6e6-48af-9106-2fbdbe0c69e0 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:31.111787 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd3b79-d88d-4880-be56-75bafc6a239e None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:31.156722 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed2042-404b-4da9-b1ac-508c3bd9b866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:31.157947 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed2042-404b-4da9-b1ac-508c3bd9b866 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:31.158613 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed2042-404b-4da9-b1ac-508c3bd9b866 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:31.159083 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed2042-404b-4da9-b1ac-508c3bd9b866 None admin] PATH_INFO: `/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:31.160496 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed2042-404b-4da9-b1ac-508c3bd9b866 None admin] RBAC: Authorizing `identity:get_project(project_id=383a1a7fd67a4b0c8baf1403a8b02f25)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:31.163176 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed2042-404b-4da9-b1ac-508c3bd9b866 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:31.165651 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 132/251] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:30 2022] GET /identity/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:31.181758 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed2042-404b-4da9-b1ac-508c3bd9b866 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:31.194994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0acc8bd-c6e6-48af-9106-2fbdbe0c69e0 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:31.245028 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6e4112-d72c-41a9-afe4-54adc79bb17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:31.245810 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e4112-d72c-41a9-afe4-54adc79bb17b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:31.246125 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e4112-d72c-41a9-afe4-54adc79bb17b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:31.246976 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e4112-d72c-41a9-afe4-54adc79bb17b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:31.247836 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6e4112-d72c-41a9-afe4-54adc79bb17b None admin] RBAC: Adding query filter params (role.id=4e60cc1f881a4caebdc208088b7ffcaa, scope.project.id=383a1a7fd67a4b0c8baf1403a8b02f25, user.id=a574aa2e7fa843dda8e77b065bb25c71) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:31.248153 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6e4112-d72c-41a9-afe4-54adc79bb17b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:31.250226 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6e4112-d72c-41a9-afe4-54adc79bb17b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:31.261402 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 120/252] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:24:31 2022] GET /identity/v3/role_assignments?user.id=a574aa2e7fa843dda8e77b065bb25c71&scope.project.id=383a1a7fd67a4b0c8baf1403a8b02f25&role.id=4e60cc1f881a4caebdc208088b7ffcaa => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:31.311016 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:31.322271 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed2042-404b-4da9-b1ac-508c3bd9b866 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:31.364134 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad883275-c6c0-4921-a516-d0f0a3488a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:31.365064 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad883275-c6c0-4921-a516-d0f0a3488a5f None admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:31.365345 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad883275-c6c0-4921-a516-d0f0a3488a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:31.365625 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad883275-c6c0-4921-a516-d0f0a3488a5f None admin] PATH_INFO: `/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25/users/e610cca9a3ba43a29f4f207fd2e1eb45/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:31.367805 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad883275-c6c0-4921-a516-d0f0a3488a5f None admin] RBAC: Authorizing `identity:create_grant(project_id=383a1a7fd67a4b0c8baf1403a8b02f25, user_id=e610cca9a3ba43a29f4f207fd2e1eb45, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:31.369304 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad883275-c6c0-4921-a516-d0f0a3488a5f None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:31.386641 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 133/253] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jun 15 19:24:31 2022] PUT /identity/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25/users/e610cca9a3ba43a29f4f207fd2e1eb45/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:24:31.412757 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f07ccd-f1ec-43cb-8c1c-f5ba9cbd96dc None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:31.413083 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f07ccd-f1ec-43cb-8c1c-f5ba9cbd96dc None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:31.413328 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f07ccd-f1ec-43cb-8c1c-f5ba9cbd96dc None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:31.416532 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 121/254] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:31.421469 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220f7cb-196b-4dc6-ba1e-0052cdb4dab3 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:31.421768 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220f7cb-196b-4dc6-ba1e-0052cdb4dab3 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:31.422237 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220f7cb-196b-4dc6-ba1e-0052cdb4dab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:31.441025 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-9220f7cb-196b-4dc6-ba1e-0052cdb4dab3 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:31.446464 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-9220f7cb-196b-4dc6-ba1e-0052cdb4dab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:31.529396 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 134/255] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 19:24:31.552245 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f07ccd-f1ec-43cb-8c1c-f5ba9cbd96dc None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:31.573129 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:31.585489 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14064f7-3832-4c10-bec5-27f1fd5cd523 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:31.587037 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14064f7-3832-4c10-bec5-27f1fd5cd523 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:31.587502 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14064f7-3832-4c10-bec5-27f1fd5cd523 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:31.589454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 135/256] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 19:24:31.595176 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1a824-1b29-4d97-8cf9-121998ac544e None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:31.595841 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1a824-1b29-4d97-8cf9-121998ac544e None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:31.596420 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1a824-1b29-4d97-8cf9-121998ac544e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:31.614292 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-9db1a824-1b29-4d97-8cf9-121998ac544e None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:31.615922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-9db1a824-1b29-4d97-8cf9-121998ac544e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:31.689597 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 136/257] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:31.725127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1a824-1b29-4d97-8cf9-121998ac544e None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:31.745919 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f07ccd-f1ec-43cb-8c1c-f5ba9cbd96dc None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:31.746533 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:31.813946 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496c6a9-64e6-4f45-93cd-ec9170e96baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:31.814723 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496c6a9-64e6-4f45-93cd-ec9170e96baa None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:31.815007 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496c6a9-64e6-4f45-93cd-ec9170e96baa None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:31.815283 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496c6a9-64e6-4f45-93cd-ec9170e96baa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:31.828140 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b496c6a9-64e6-4f45-93cd-ec9170e96baa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:31.838130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-b496c6a9-64e6-4f45-93cd-ec9170e96baa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 19:24:31.838130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:31.838130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:31.838130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:31.838130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:31.838130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:31.838130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:31.838130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:31.838130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:31.838130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:31.838130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:31.838130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:31.838130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:31.838130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:31.838130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:31.838130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:31.839033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:31.839033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:31.839033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:31.839033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:31.839033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:31.839033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:31.839033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:31.839033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:31.839033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:31.839033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:31.839033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:31.839033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:31.839033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:31.839033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:31.839033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:31.839033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:31.839845 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:31.839845 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:31.839845 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:31.839845 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:31.839845 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:31.839845 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:31.839845 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:31.839845 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:31.839845 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:31.839845 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:31.839845 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 19:24:31.839845 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:24:31.839845 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 122/258] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jun 15 19:24:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:31.850178 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496c6a9-64e6-4f45-93cd-ec9170e96baa None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:31.883030 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1a824-1b29-4d97-8cf9-121998ac544e None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:31.932018 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcccaf0-0a59-4b5c-8a66-4dca0550ceb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:31.933332 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcccaf0-0a59-4b5c-8a66-4dca0550ceb0 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:31.933808 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcccaf0-0a59-4b5c-8a66-4dca0550ceb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:31.934358 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcccaf0-0a59-4b5c-8a66-4dca0550ceb0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:31.952102 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:31.961666 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcccaf0-0a59-4b5c-8a66-4dca0550ceb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:31.970034 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-7bcccaf0-0a59-4b5c-8a66-4dca0550ceb0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:24:31.970034 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:31.970034 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:31.970034 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:31.970034 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:31.970034 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:31.970034 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:31.970034 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:31.970034 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:31.970034 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:31.970034 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:31.970034 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:31.970034 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:31.970034 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:31.970034 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:31.970034 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:31.970927 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:31.970927 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:31.970927 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:31.970927 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:31.970927 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:31.970927 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:31.970927 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:31.970927 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:31.970927 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:31.970927 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:31.970927 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:31.970927 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:31.970927 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:31.970927 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:31.970927 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:31.970927 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:31.971717 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:31.971717 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:31.971717 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:31.971717 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:31.971717 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:31.971717 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:31.971717 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:31.971717 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:31.971717 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:31.971717 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:31.971717 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:24:31.971717 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:24:31.974192 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 137/259] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jun 15 19:24:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:24:31.981668 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcccaf0-0a59-4b5c-8a66-4dca0550ceb0 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:32.029169 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496c6a9-64e6-4f45-93cd-ec9170e96baa None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:32.085437 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9fef5-5366-4bb9-8933-e90a5021833e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:32.094233 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9fef5-5366-4bb9-8933-e90a5021833e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:32.094233 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9fef5-5366-4bb9-8933-e90a5021833e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:32.094233 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9fef5-5366-4bb9-8933-e90a5021833e None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:32.094233 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f9fef5-5366-4bb9-8933-e90a5021833e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:32.094233 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f9fef5-5366-4bb9-8933-e90a5021833e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:32.094233 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f9fef5-5366-4bb9-8933-e90a5021833e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:32.095325 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:32.114402 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 123/260] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:24:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:32.119816 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9fef5-5366-4bb9-8933-e90a5021833e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:32.155691 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcccaf0-0a59-4b5c-8a66-4dca0550ceb0 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:32.228793 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57177763-9718-4759-9d84-e3eccd124b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:32.229474 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57177763-9718-4759-9d84-e3eccd124b33 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:32.229870 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57177763-9718-4759-9d84-e3eccd124b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:32.230250 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57177763-9718-4759-9d84-e3eccd124b33 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:32.231157 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57177763-9718-4759-9d84-e3eccd124b33 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:32.231472 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57177763-9718-4759-9d84-e3eccd124b33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:32.232880 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57177763-9718-4759-9d84-e3eccd124b33 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:32.248679 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 138/261] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:24:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:24:32.254274 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57177763-9718-4759-9d84-e3eccd124b33 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:32.290242 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9fef5-5366-4bb9-8933-e90a5021833e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:32.334477 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:32.340667 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7add5-97f8-4590-abe6-f0d800c46720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:32.341794 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7add5-97f8-4590-abe6-f0d800c46720 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:32.342341 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7add5-97f8-4590-abe6-f0d800c46720 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:32.342840 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7add5-97f8-4590-abe6-f0d800c46720 None admin] PATH_INFO: `/v3/users/1041c9fb868f414798a4a50b20e459af` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:32.346934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab7add5-97f8-4590-abe6-f0d800c46720 None admin] RBAC: Authorizing `identity:get_user(user_id=1041c9fb868f414798a4a50b20e459af)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:32.349394 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab7add5-97f8-4590-abe6-f0d800c46720 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:32.351439 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 124/262] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:24:32 2022] GET /identity/v3/users/1041c9fb868f414798a4a50b20e459af => generated 283 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:24:32.362834 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7add5-97f8-4590-abe6-f0d800c46720 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:32.488214 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57177763-9718-4759-9d84-e3eccd124b33 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:32.549877 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ba22f-7fc0-4280-a397-f68363b9317c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:32.550776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ba22f-7fc0-4280-a397-f68363b9317c None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:32.551054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ba22f-7fc0-4280-a397-f68363b9317c None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:32.551327 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ba22f-7fc0-4280-a397-f68363b9317c None admin] PATH_INFO: `/v3/users/1041c9fb868f414798a4a50b20e459af` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:32.556362 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ba22f-7fc0-4280-a397-f68363b9317c None admin] RBAC: Authorizing `identity:get_user(user_id=1041c9fb868f414798a4a50b20e459af)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:32.557808 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ba22f-7fc0-4280-a397-f68363b9317c None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:32.559492 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 139/263] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:24:32 2022] GET /identity/v3/users/1041c9fb868f414798a4a50b20e459af => generated 283 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:24:32.564724 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ba22f-7fc0-4280-a397-f68363b9317c None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:32.570379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:32.572960 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7add5-97f8-4590-abe6-f0d800c46720 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:32.607873 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4132a-a369-41e3-864c-f763a9ac1303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:32.608457 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4132a-a369-41e3-864c-f763a9ac1303 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:32.608739 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4132a-a369-41e3-864c-f763a9ac1303 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:32.609011 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4132a-a369-41e3-864c-f763a9ac1303 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:32.675970 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:32.686370 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b4132a-a369-41e3-864c-f763a9ac1303 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:32.687845 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b4132a-a369-41e3-864c-f763a9ac1303 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:32.778637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ba22f-7fc0-4280-a397-f68363b9317c None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:32.842502 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 125/264] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:24:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 19:24:32.848253 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4132a-a369-41e3-864c-f763a9ac1303 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:32.854042 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556dda96-45c7-4901-9a5c-27a0d01aaef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:32.854727 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556dda96-45c7-4901-9a5c-27a0d01aaef4 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:32.855011 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556dda96-45c7-4901-9a5c-27a0d01aaef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:32.855289 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556dda96-45c7-4901-9a5c-27a0d01aaef4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:32.992468 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556dda96-45c7-4901-9a5c-27a0d01aaef4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:32.994310 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556dda96-45c7-4901-9a5c-27a0d01aaef4 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:33.002117 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:33.020431 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:33.025771 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4132a-a369-41e3-864c-f763a9ac1303 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:33.098798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e0f3f-dbc4-4235-b458-bee90f2303ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:33.099671 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e0f3f-dbc4-4235-b458-bee90f2303ff None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:33.099958 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e0f3f-dbc4-4235-b458-bee90f2303ff None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:33.100235 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e0f3f-dbc4-4235-b458-bee90f2303ff None admin] PATH_INFO: `/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:33.101550 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987e0f3f-dbc4-4235-b458-bee90f2303ff None admin] RBAC: Authorizing `identity:get_project(project_id=383a1a7fd67a4b0c8baf1403a8b02f25)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:33.102945 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987e0f3f-dbc4-4235-b458-bee90f2303ff None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:33.104972 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 126/265] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:32 2022] GET /identity/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25 => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:33.110935 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e0f3f-dbc4-4235-b458-bee90f2303ff None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:33.175291 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 140/266] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:24:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 19:24:33.180896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556dda96-45c7-4901-9a5c-27a0d01aaef4 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:33.298745 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e0f3f-dbc4-4235-b458-bee90f2303ff None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:33.303671 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556dda96-45c7-4901-9a5c-27a0d01aaef4 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:33.306638 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:33.339531 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b973f32-d262-463a-a199-c2b4f4235223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:33.340778 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b973f32-d262-463a-a199-c2b4f4235223 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:33.341712 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b973f32-d262-463a-a199-c2b4f4235223 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:33.342228 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b973f32-d262-463a-a199-c2b4f4235223 None admin] PATH_INFO: `/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:33.343693 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b973f32-d262-463a-a199-c2b4f4235223 None admin] RBAC: Authorizing `identity:get_project(project_id=383a1a7fd67a4b0c8baf1403a8b02f25)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:33.345350 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b973f32-d262-463a-a199-c2b4f4235223 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:33.347970 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 141/267] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:33 2022] GET /identity/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:33.355066 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b973f32-d262-463a-a199-c2b4f4235223 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:33.442809 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5ad79-289d-44fc-86dd-260c8e2a18cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:33.443484 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5ad79-289d-44fc-86dd-260c8e2a18cd None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:33.443770 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5ad79-289d-44fc-86dd-260c8e2a18cd None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:33.444047 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5ad79-289d-44fc-86dd-260c8e2a18cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:33.444919 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a5ad79-289d-44fc-86dd-260c8e2a18cd None admin] RBAC: Adding query filter params (role.id=3920d5126e22480ab9a4eec9cf0fd4e7, scope.project.id=383a1a7fd67a4b0c8baf1403a8b02f25, user.id=1041c9fb868f414798a4a50b20e459af) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:33.445214 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a5ad79-289d-44fc-86dd-260c8e2a18cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:33.446704 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a5ad79-289d-44fc-86dd-260c8e2a18cd None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:33.468061 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 127/268] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:24:33 2022] GET /identity/v3/role_assignments?user.id=1041c9fb868f414798a4a50b20e459af&scope.project.id=383a1a7fd67a4b0c8baf1403a8b02f25&role.id=3920d5126e22480ab9a4eec9cf0fd4e7 => generated 608 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:33.580924 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b973f32-d262-463a-a199-c2b4f4235223 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:33.633445 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e29d4a1-95ec-4823-a9d8-84b1c484b708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:33.634784 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29d4a1-95ec-4823-a9d8-84b1c484b708 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:33.635271 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29d4a1-95ec-4823-a9d8-84b1c484b708 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:33.635749 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29d4a1-95ec-4823-a9d8-84b1c484b708 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:33.636798 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e29d4a1-95ec-4823-a9d8-84b1c484b708 None admin] RBAC: Adding query filter params (role.id=4f736313c26d4f8db48bbb2f346e50e4, scope.project.id=383a1a7fd67a4b0c8baf1403a8b02f25, user.id=1041c9fb868f414798a4a50b20e459af) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:33.637423 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e29d4a1-95ec-4823-a9d8-84b1c484b708 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:33.639154 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e29d4a1-95ec-4823-a9d8-84b1c484b708 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:33.646453 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 142/269] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:24:33 2022] GET /identity/v3/role_assignments?user.id=1041c9fb868f414798a4a50b20e459af&scope.project.id=383a1a7fd67a4b0c8baf1403a8b02f25&role.id=4f736313c26d4f8db48bbb2f346e50e4 => generated 608 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:34.092671 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab7ce8e-9ede-4eaf-ba78-f4b8421e6b77 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:34.093688 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab7ce8e-9ede-4eaf-ba78-f4b8421e6b77 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:34.094238 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab7ce8e-9ede-4eaf-ba78-f4b8421e6b77 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:34.095849 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 128/270] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:34.101647 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef244d-c4f9-4608-bdba-b5ef61c0d81b None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:34.102072 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef244d-c4f9-4608-bdba-b5ef61c0d81b None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:34.102280 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef244d-c4f9-4608-bdba-b5ef61c0d81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:34.123466 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-feef244d-c4f9-4608-bdba-b5ef61c0d81b None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:34.124653 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-feef244d-c4f9-4608-bdba-b5ef61c0d81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:34.156283 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:34.207933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 143/271] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 19:24:34.230711 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab7ce8e-9ede-4eaf-ba78-f4b8421e6b77 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:34.273592 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:34.284837 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3d226-7db8-4b84-9525-ffa20487c1d9 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:34.285272 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3d226-7db8-4b84-9525-ffa20487c1d9 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:34.285651 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3d226-7db8-4b84-9525-ffa20487c1d9 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:34.287434 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 144/272] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:34.292466 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d217e6-5ae7-421d-9b5b-f66d97b0edf8 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:34.292752 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d217e6-5ae7-421d-9b5b-f66d97b0edf8 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:34.293027 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d217e6-5ae7-421d-9b5b-f66d97b0edf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:34.311919 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-d2d217e6-5ae7-421d-9b5b-f66d97b0edf8 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:34.313196 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-d2d217e6-5ae7-421d-9b5b-f66d97b0edf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:34.379238 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab7ce8e-9ede-4eaf-ba78-f4b8421e6b77 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:34.398862 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 145/273] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 19:24:34.423615 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d217e6-5ae7-421d-9b5b-f66d97b0edf8 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:34.424773 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b5c1a-5e68-4522-9398-00e177f74074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:34.426012 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b5c1a-5e68-4522-9398-00e177f74074 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:34.427402 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b5c1a-5e68-4522-9398-00e177f74074 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:34.427946 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b5c1a-5e68-4522-9398-00e177f74074 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:34.441977 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3b5c1a-5e68-4522-9398-00e177f74074 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:34.450081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-ec3b5c1a-5e68-4522-9398-00e177f74074 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 19:24:34.450081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:34.450081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:34.450081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:34.450081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:34.450081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:34.450081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:34.450081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:34.450081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:34.450081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:34.450081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:34.450081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:34.450081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:34.450081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:34.450081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:34.450081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:34.451096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:34.451096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:34.451096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:34.451096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:34.451096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:34.451096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:34.451096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:34.451096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:34.451096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:34.451096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:34.451096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:34.451096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:34.451096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:34.451096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:34.451096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:34.451096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:34.451933 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:34.451933 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:34.451933 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:34.451933 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:34.451933 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:34.451933 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:34.451933 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:34.451933 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:34.451933 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:34.451933 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:34.451933 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 19:24:34.451933 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:24:34.451933 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 129/274] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:24:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:34.452815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61839fa5-a8be-45c8-9b52-ee55638f6942 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:34.453061 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61839fa5-a8be-45c8-9b52-ee55638f6942 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:34.453343 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61839fa5-a8be-45c8-9b52-ee55638f6942 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:34.454971 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 130/275] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:34.459757 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61839fa5-a8be-45c8-9b52-ee55638f6942 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:34.539505 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:34.547055 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d217e6-5ae7-421d-9b5b-f66d97b0edf8 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:34.552544 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:34.563227 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61839fa5-a8be-45c8-9b52-ee55638f6942 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:34.579606 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b7a3a1-96d5-45d0-8043-9e31125ad929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:34.580401 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7a3a1-96d5-45d0-8043-9e31125ad929 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:34.580684 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7a3a1-96d5-45d0-8043-9e31125ad929 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:34.580961 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7a3a1-96d5-45d0-8043-9e31125ad929 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:34.582162 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b7a3a1-96d5-45d0-8043-9e31125ad929 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:34.583614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b7a3a1-96d5-45d0-8043-9e31125ad929 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:34.585247 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 146/276] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:24:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:24:34.586407 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f6a98-e91d-4576-9bdc-100caff65866 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:34.586693 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f6a98-e91d-4576-9bdc-100caff65866 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:34.587059 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f6a98-e91d-4576-9bdc-100caff65866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:34.598191 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64268db5-93bd-4cba-9e95-9c78d04d5db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:34.598859 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64268db5-93bd-4cba-9e95-9c78d04d5db4 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:34.599138 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64268db5-93bd-4cba-9e95-9c78d04d5db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:34.599410 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64268db5-93bd-4cba-9e95-9c78d04d5db4 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:34.600158 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64268db5-93bd-4cba-9e95-9c78d04d5db4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:34.600459 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64268db5-93bd-4cba-9e95-9c78d04d5db4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:34.601835 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64268db5-93bd-4cba-9e95-9c78d04d5db4 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:34.603126 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-ee8f6a98-e91d-4576-9bdc-100caff65866 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:34.604297 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-ee8f6a98-e91d-4576-9bdc-100caff65866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:34.611096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 131/277] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jun 15 19:24:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:24:34.612531 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f228fe-80a2-44e8-a7a9-0db0103d5c21 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:34.612757 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f228fe-80a2-44e8-a7a9-0db0103d5c21 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:34.613029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f228fe-80a2-44e8-a7a9-0db0103d5c21 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:34.614558 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 132/278] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:34.615128 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f228fe-80a2-44e8-a7a9-0db0103d5c21 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:34.671255 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 147/279] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:34.672025 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f6a98-e91d-4576-9bdc-100caff65866 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:34.719766 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f228fe-80a2-44e8-a7a9-0db0103d5c21 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:34.754000 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660694f-dd93-4a79-9df1-58e35cbad0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:34.754719 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660694f-dd93-4a79-9df1-58e35cbad0a5 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:34.755003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660694f-dd93-4a79-9df1-58e35cbad0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:34.755279 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660694f-dd93-4a79-9df1-58e35cbad0a5 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:34.755990 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e660694f-dd93-4a79-9df1-58e35cbad0a5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:34.757433 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e660694f-dd93-4a79-9df1-58e35cbad0a5 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:34.761077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:34.772082 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f6a98-e91d-4576-9bdc-100caff65866 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:34.777059 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 133/280] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Jun 15 19:24:34 2022] POST /identity/v3/users => generated 293 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:24:34.777059 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9fe97f-ee2f-45e2-87cf-575ed7a8dd2a None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:34.777059 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9fe97f-ee2f-45e2-87cf-575ed7a8dd2a None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:34.777059 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9fe97f-ee2f-45e2-87cf-575ed7a8dd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:34.797198 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-1f9fe97f-ee2f-45e2-87cf-575ed7a8dd2a None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:34.798467 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1f9fe97f-ee2f-45e2-87cf-575ed7a8dd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:34.800941 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:34.835189 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e908be-2b97-4bd4-a70b-8e5aeb7ab02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:34.836500 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e908be-2b97-4bd4-a70b-8e5aeb7ab02f None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:34.836984 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e908be-2b97-4bd4-a70b-8e5aeb7ab02f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:34.837775 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e908be-2b97-4bd4-a70b-8e5aeb7ab02f None admin] PATH_INFO: `/v3/users/a574aa2e7fa843dda8e77b065bb25c71` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:34.841337 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e908be-2b97-4bd4-a70b-8e5aeb7ab02f None admin] RBAC: Authorizing `identity:get_user(user_id=a574aa2e7fa843dda8e77b065bb25c71)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:34.843567 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e908be-2b97-4bd4-a70b-8e5aeb7ab02f None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:34.846440 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 148/281] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:24:34 2022] GET /identity/v3/users/a574aa2e7fa843dda8e77b065bb25c71 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:34.851498 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e908be-2b97-4bd4-a70b-8e5aeb7ab02f None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:34.873528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 134/282] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:34.874517 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9fe97f-ee2f-45e2-87cf-575ed7a8dd2a None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:34.988839 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e908be-2b97-4bd4-a70b-8e5aeb7ab02f None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:34.995117 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9fe97f-ee2f-45e2-87cf-575ed7a8dd2a None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:35.034131 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513b4ca-5632-4cc8-851f-cde5e3dcf4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:35.035269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513b4ca-5632-4cc8-851f-cde5e3dcf4f7 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:35.035758 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513b4ca-5632-4cc8-851f-cde5e3dcf4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:35.036136 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513b4ca-5632-4cc8-851f-cde5e3dcf4f7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:35.039987 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc93980-5344-4ab0-b8e9-37b8fedeb6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:35.041153 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc93980-5344-4ab0-b8e9-37b8fedeb6f9 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:35.041432 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc93980-5344-4ab0-b8e9-37b8fedeb6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:35.041710 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc93980-5344-4ab0-b8e9-37b8fedeb6f9 None admin] PATH_INFO: `/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:35.043059 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc93980-5344-4ab0-b8e9-37b8fedeb6f9 None admin] RBAC: Authorizing `identity:get_project(project_id=383a1a7fd67a4b0c8baf1403a8b02f25)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:35.044877 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc93980-5344-4ab0-b8e9-37b8fedeb6f9 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:35.047092 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 135/283] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:34 2022] GET /identity/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:35.048269 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc93980-5344-4ab0-b8e9-37b8fedeb6f9 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:35.054195 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c513b4ca-5632-4cc8-851f-cde5e3dcf4f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:35.061105 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:35.062793 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-c513b4ca-5632-4cc8-851f-cde5e3dcf4f7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 19:24:35.062793 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:35.062793 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:35.062793 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:35.062793 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:35.062793 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:35.062793 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:35.062793 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:35.062793 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:35.062793 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:35.062793 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:35.062793 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:35.062793 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:35.062793 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:35.062793 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:35.062793 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:35.063688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:35.063688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:35.063688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:35.063688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:35.063688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:35.063688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:35.063688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:35.063688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:35.063688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:35.063688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:35.063688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:35.063688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:35.063688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:35.063688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:35.063688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:35.063688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:35.064485 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:35.064485 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:35.064485 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:35.064485 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:35.064485 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:35.064485 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:35.064485 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:35.064485 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:35.064485 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:35.064485 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:35.064485 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 19:24:35.064485 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:24:35.064485 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 149/284] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:24:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:35.065134 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513b4ca-5632-4cc8-851f-cde5e3dcf4f7 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:35.094469 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:35.180638 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc93980-5344-4ab0-b8e9-37b8fedeb6f9 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:35.210202 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513b4ca-5632-4cc8-851f-cde5e3dcf4f7 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:35.218994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7dc59a-0ca3-4753-8f8b-e934e713d5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:35.220328 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7dc59a-0ca3-4753-8f8b-e934e713d5c5 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:35.220328 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7dc59a-0ca3-4753-8f8b-e934e713d5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:35.220625 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7dc59a-0ca3-4753-8f8b-e934e713d5c5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:35.234737 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7dc59a-0ca3-4753-8f8b-e934e713d5c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:35.243962 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-9d7dc59a-0ca3-4753-8f8b-e934e713d5c5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 19:24:35.243962 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:35.243962 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:35.243962 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:35.243962 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:35.243962 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:35.243962 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:35.243962 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:35.243962 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:35.243962 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:35.243962 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:35.243962 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:35.243962 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:35.243962 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:35.243962 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:35.243962 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:35.245055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:35.245055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:35.245055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:35.245055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:35.245055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:35.245055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:35.245055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:35.245055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:35.245055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:35.245055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:35.245055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:35.245055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:35.245055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:35.245055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:35.245055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:35.245055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:35.246008 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:35.246008 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:35.246008 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:35.246008 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:35.246008 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:35.246008 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:35.246008 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:35.246008 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:35.246008 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:35.246008 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:35.246008 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 19:24:35.246008 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:24:35.246008 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 136/285] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:24:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:35.250839 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7dc59a-0ca3-4753-8f8b-e934e713d5c5 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:35.254427 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b8fc03-5bd4-47cd-a95e-d388d84692d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:35.255533 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8fc03-5bd4-47cd-a95e-d388d84692d8 None admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:35.257104 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8fc03-5bd4-47cd-a95e-d388d84692d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:35.257528 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8fc03-5bd4-47cd-a95e-d388d84692d8 None admin] PATH_INFO: `/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25/users/a574aa2e7fa843dda8e77b065bb25c71/roles/4e60cc1f881a4caebdc208088b7ffcaa` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:35.260002 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b8fc03-5bd4-47cd-a95e-d388d84692d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=383a1a7fd67a4b0c8baf1403a8b02f25, user_id=a574aa2e7fa843dda8e77b065bb25c71, role_id=4e60cc1f881a4caebdc208088b7ffcaa)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:35.261603 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b8fc03-5bd4-47cd-a95e-d388d84692d8 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:35.279198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 150/286] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jun 15 19:24:35 2022] PUT /identity/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25/users/a574aa2e7fa843dda8e77b065bb25c71/roles/4e60cc1f881a4caebdc208088b7ffcaa => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:24:35.287456 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b8fc03-5bd4-47cd-a95e-d388d84692d8 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:35.375675 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:35.443991 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:35.444364 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7dc59a-0ca3-4753-8f8b-e934e713d5c5 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:35.487131 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b8fc03-5bd4-47cd-a95e-d388d84692d8 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:35.510938 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2a0b2-757e-4345-9d07-689837bf5329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:35.511537 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2a0b2-757e-4345-9d07-689837bf5329 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:35.511823 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2a0b2-757e-4345-9d07-689837bf5329 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:35.512136 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2a0b2-757e-4345-9d07-689837bf5329 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:35.512964 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2a0b2-757e-4345-9d07-689837bf5329 None admin] RBAC: Adding query filter params (name=member) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:35.513272 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2a0b2-757e-4345-9d07-689837bf5329 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:35.514738 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2a0b2-757e-4345-9d07-689837bf5329 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:35.525186 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 137/287] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jun 15 19:24:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:35.530497 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2a0b2-757e-4345-9d07-689837bf5329 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:35.533753 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8dd60d-dfbe-4925-8511-1df3b32b5f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:35.534697 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8dd60d-dfbe-4925-8511-1df3b32b5f66 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:35.535231 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8dd60d-dfbe-4925-8511-1df3b32b5f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:35.535704 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8dd60d-dfbe-4925-8511-1df3b32b5f66 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:35.536660 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8dd60d-dfbe-4925-8511-1df3b32b5f66 None admin] RBAC: Adding query filter params (name=member) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:35.537179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8dd60d-dfbe-4925-8511-1df3b32b5f66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:35.538904 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8dd60d-dfbe-4925-8511-1df3b32b5f66 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:35.548742 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 151/288] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jun 15 19:24:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:35.554251 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8dd60d-dfbe-4925-8511-1df3b32b5f66 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:35.639748 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2a0b2-757e-4345-9d07-689837bf5329 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:35.666705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8dd60d-dfbe-4925-8511-1df3b32b5f66 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:35.667598 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:35.676235 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a6c0b0-447a-4b35-872b-fd9c362f0370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:35.677529 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a6c0b0-447a-4b35-872b-fd9c362f0370 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:35.677529 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a6c0b0-447a-4b35-872b-fd9c362f0370 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:35.677724 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a6c0b0-447a-4b35-872b-fd9c362f0370 None admin] PATH_INFO: `/v3/users/e610cca9a3ba43a29f4f207fd2e1eb45` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:35.679077 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a6c0b0-447a-4b35-872b-fd9c362f0370 None admin] RBAC: Authorizing `identity:get_user(user_id=e610cca9a3ba43a29f4f207fd2e1eb45)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:35.680531 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a6c0b0-447a-4b35-872b-fd9c362f0370 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:35.686941 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 138/289] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:24:35 2022] GET /identity/v3/users/e610cca9a3ba43a29f4f207fd2e1eb45 => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:35.688596 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a6c0b0-447a-4b35-872b-fd9c362f0370 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:35.708945 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef93436-2204-4e7e-9793-b7c894236ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:35.710266 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef93436-2204-4e7e-9793-b7c894236ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:35.710755 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef93436-2204-4e7e-9793-b7c894236ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:35.711255 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef93436-2204-4e7e-9793-b7c894236ab6 None admin] PATH_INFO: `/v3/users/bc387c006e2047878b5ccb05a452a2c6` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:35.724145 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:35.727615 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef93436-2204-4e7e-9793-b7c894236ab6 None admin] RBAC: Authorizing `identity:get_user(user_id=bc387c006e2047878b5ccb05a452a2c6)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:35.729353 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef93436-2204-4e7e-9793-b7c894236ab6 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:35.731804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 152/290] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:24:35 2022] GET /identity/v3/users/bc387c006e2047878b5ccb05a452a2c6 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:24:35.739403 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef93436-2204-4e7e-9793-b7c894236ab6 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:35.798044 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a6c0b0-447a-4b35-872b-fd9c362f0370 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:35.848726 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c196334-4675-48bd-adbf-78193a93a2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:35.849382 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c196334-4675-48bd-adbf-78193a93a2c6 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:35.849664 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c196334-4675-48bd-adbf-78193a93a2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:35.849940 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c196334-4675-48bd-adbf-78193a93a2c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:35.888780 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef93436-2204-4e7e-9793-b7c894236ab6 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:35.943270 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:35.956003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c196334-4675-48bd-adbf-78193a93a2c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:35.957611 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c196334-4675-48bd-adbf-78193a93a2c6 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:35.960572 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86de619-ddf1-44a2-92b2-5865ba6d19ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:35.961235 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86de619-ddf1-44a2-92b2-5865ba6d19ed None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:35.961513 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86de619-ddf1-44a2-92b2-5865ba6d19ed None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:35.961788 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86de619-ddf1-44a2-92b2-5865ba6d19ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:35.962645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86de619-ddf1-44a2-92b2-5865ba6d19ed None admin] RBAC: Adding query filter params (role.id=e53a78912e764f8387e0cf637bb86bee, user.id=bc387c006e2047878b5ccb05a452a2c6, scope.system=all) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:35.962953 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86de619-ddf1-44a2-92b2-5865ba6d19ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:35.964361 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86de619-ddf1-44a2-92b2-5865ba6d19ed None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:35.970551 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 153/291] 10.222.0.15 () {56 vars in 1308 bytes} [Wed Jun 15 19:24:35 2022] GET /identity/v3/role_assignments?user.id=bc387c006e2047878b5ccb05a452a2c6&scope.system=all&role.id=e53a78912e764f8387e0cf637bb86bee => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:36.080798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 139/292] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:24:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 19:24:36.086815 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86de619-ddf1-44a2-92b2-5865ba6d19ed None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:36.193165 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:36.247061 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86de619-ddf1-44a2-92b2-5865ba6d19ed None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:36.283560 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6be58-5b06-4d3c-a193-f9936e895437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:36.284726 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6be58-5b06-4d3c-a193-f9936e895437 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:36.285185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6be58-5b06-4d3c-a193-f9936e895437 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:36.285663 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6be58-5b06-4d3c-a193-f9936e895437 None admin] PATH_INFO: `/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:36.287062 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6be58-5b06-4d3c-a193-f9936e895437 None admin] RBAC: Authorizing `identity:get_project(project_id=383a1a7fd67a4b0c8baf1403a8b02f25)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:36.288666 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6be58-5b06-4d3c-a193-f9936e895437 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:36.290585 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 154/293] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:36 2022] GET /identity/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:36.297264 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c196334-4675-48bd-adbf-78193a93a2c6 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:36.456219 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:36.466475 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c196334-4675-48bd-adbf-78193a93a2c6 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:36.512028 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3a5fb-6eaf-4e7f-b4d0-33a52d139b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:36.512687 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3a5fb-6eaf-4e7f-b4d0-33a52d139b3f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:36.512969 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3a5fb-6eaf-4e7f-b4d0-33a52d139b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:36.513284 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3a5fb-6eaf-4e7f-b4d0-33a52d139b3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:36.514135 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d3a5fb-6eaf-4e7f-b4d0-33a52d139b3f None admin] RBAC: Adding query filter params (role.id=e53a78912e764f8387e0cf637bb86bee, scope.project.id=383a1a7fd67a4b0c8baf1403a8b02f25, user.id=e610cca9a3ba43a29f4f207fd2e1eb45) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:36.514481 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d3a5fb-6eaf-4e7f-b4d0-33a52d139b3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:36.515869 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d3a5fb-6eaf-4e7f-b4d0-33a52d139b3f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:36.525304 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 140/294] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:24:36 2022] GET /identity/v3/role_assignments?user.id=e610cca9a3ba43a29f4f207fd2e1eb45&scope.project.id=383a1a7fd67a4b0c8baf1403a8b02f25&role.id=e53a78912e764f8387e0cf637bb86bee => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:37.287768 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e0616-35a1-49e8-816e-d6e504ade86e None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:37.287768 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e0616-35a1-49e8-816e-d6e504ade86e None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:37.288487 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e0616-35a1-49e8-816e-d6e504ade86e None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:37.289672 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 155/295] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:37.294722 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449d10b-1036-4fdd-99ef-448937b810e7 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:37.295014 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449d10b-1036-4fdd-99ef-448937b810e7 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:37.295291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449d10b-1036-4fdd-99ef-448937b810e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:37.312446 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c449d10b-1036-4fdd-99ef-448937b810e7 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:37.313617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c449d10b-1036-4fdd-99ef-448937b810e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:37.396560 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 141/296] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:37.412437 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbde1d1-c658-45df-b05b-d4b41d88a417 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:37.412990 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbde1d1-c658-45df-b05b-d4b41d88a417 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:37.413468 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbde1d1-c658-45df-b05b-d4b41d88a417 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:37.415378 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 156/297] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:37.419522 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c449d10b-1036-4fdd-99ef-448937b810e7 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:37.420909 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117de15-1dda-4645-b0c5-7ff6021390ec None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:37.421202 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117de15-1dda-4645-b0c5-7ff6021390ec None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:37.421484 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117de15-1dda-4645-b0c5-7ff6021390ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:37.444141 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-7117de15-1dda-4645-b0c5-7ff6021390ec None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:37.445635 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-7117de15-1dda-4645-b0c5-7ff6021390ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:37.538244 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 157/298] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:37.548969 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:37.551147 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c449d10b-1036-4fdd-99ef-448937b810e7 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:37.565186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117de15-1dda-4645-b0c5-7ff6021390ec None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:37.596935 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4c543-c802-40cb-8f76-6d7239422f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:37.598596 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4c543-c802-40cb-8f76-6d7239422f19 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:37.599135 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4c543-c802-40cb-8f76-6d7239422f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:37.599613 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4c543-c802-40cb-8f76-6d7239422f19 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:37.616527 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f4c543-c802-40cb-8f76-6d7239422f19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:37.627489 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-00f4c543-c802-40cb-8f76-6d7239422f19 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 19:24:37.627489 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:37.627489 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:37.627489 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:37.627489 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:37.627489 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:37.627489 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:37.627489 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:37.627489 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:37.627489 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:37.627489 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:37.627489 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:37.627489 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:37.627489 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:37.627489 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:37.627489 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:37.628574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:37.628574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:37.628574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:37.628574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:37.628574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:37.628574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:37.628574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:37.628574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:37.628574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:37.628574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:37.628574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:37.628574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:37.628574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:37.628574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:37.628574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:37.628574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:37.629593 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:37.629593 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:37.629593 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:37.629593 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:37.629593 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:37.629593 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:37.629593 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:37.629593 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:37.629593 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:37.629593 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:37.629593 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 19:24:37.629593 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:24:37.630981 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:37.632880 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 142/299] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:24:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:37.639396 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4c543-c802-40cb-8f76-6d7239422f19 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:37.678293 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117de15-1dda-4645-b0c5-7ff6021390ec None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:37.715589 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3284e31d-9f45-422b-8188-f39129838a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:37.716629 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284e31d-9f45-422b-8188-f39129838a59 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:37.717054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284e31d-9f45-422b-8188-f39129838a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:37.717454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284e31d-9f45-422b-8188-f39129838a59 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:37.718713 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3284e31d-9f45-422b-8188-f39129838a59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:37.720351 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3284e31d-9f45-422b-8188-f39129838a59 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:37.722678 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 158/300] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:24:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:24:37.728039 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3284e31d-9f45-422b-8188-f39129838a59 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:37.764959 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4c543-c802-40cb-8f76-6d7239422f19 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:37.812763 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47e889-6b4a-4194-ae3c-9290dcd8b3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:37.813684 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47e889-6b4a-4194-ae3c-9290dcd8b3f4 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:37.814153 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47e889-6b4a-4194-ae3c-9290dcd8b3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:37.815165 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47e889-6b4a-4194-ae3c-9290dcd8b3f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:37.816088 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47e889-6b4a-4194-ae3c-9290dcd8b3f4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:37.817037 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47e889-6b4a-4194-ae3c-9290dcd8b3f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:37.819165 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47e889-6b4a-4194-ae3c-9290dcd8b3f4 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:37.831531 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 143/301] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jun 15 19:24:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:37.837435 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47e889-6b4a-4194-ae3c-9290dcd8b3f4 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:37.844810 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3284e31d-9f45-422b-8188-f39129838a59 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:37.857269 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:37.878940 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:37.886035 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3210e-4346-4222-b68c-cca1f2d06ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:37.886739 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3210e-4346-4222-b68c-cca1f2d06ee6 None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:37.887054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3210e-4346-4222-b68c-cca1f2d06ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:37.887337 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3210e-4346-4222-b68c-cca1f2d06ee6 None admin] PATH_INFO: `/v3/groups` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:37.888153 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d3210e-4346-4222-b68c-cca1f2d06ee6 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:37.889667 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d3210e-4346-4222-b68c-cca1f2d06ee6 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:37.914126 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 159/302] 10.222.0.15 () {60 vars in 1134 bytes} [Wed Jun 15 19:24:37 2022] POST /identity/v3/groups => generated 229 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:24:37.978366 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47e889-6b4a-4194-ae3c-9290dcd8b3f4 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:38.015878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275d0e5-896a-4cbf-97e8-fd4dc4562e14 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:38.016231 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275d0e5-896a-4cbf-97e8-fd4dc4562e14 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:38.016446 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275d0e5-896a-4cbf-97e8-fd4dc4562e14 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:38.017914 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42130442-a889-4393-933b-85c84d3689eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:38.018137 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 160/303] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 19:24:38.018891 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42130442-a889-4393-933b-85c84d3689eb None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:38.019180 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42130442-a889-4393-933b-85c84d3689eb None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:38.019460 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42130442-a889-4393-933b-85c84d3689eb None admin] PATH_INFO: `/v3/users/952833e57cac4cf88e4d8e61c0f071e1` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:38.023717 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258f3a3-4e30-424f-9725-ac332df84cd8 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:38.024011 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258f3a3-4e30-424f-9725-ac332df84cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:38.024296 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258f3a3-4e30-424f-9725-ac332df84cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:38.037357 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42130442-a889-4393-933b-85c84d3689eb None admin] RBAC: Authorizing `identity:get_user(user_id=952833e57cac4cf88e4d8e61c0f071e1)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:38.039268 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42130442-a889-4393-933b-85c84d3689eb None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:38.041301 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 144/304] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:24:37 2022] GET /identity/v3/users/952833e57cac4cf88e4d8e61c0f071e1 => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:38.042397 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-7258f3a3-4e30-424f-9725-ac332df84cd8 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:38.043685 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-7258f3a3-4e30-424f-9725-ac332df84cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:38.051159 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42130442-a889-4393-933b-85c84d3689eb None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:38.122912 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 161/305] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 19:24:38.132337 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:38.141960 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7258f3a3-4e30-424f-9725-ac332df84cd8 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:38.173176 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42130442-a889-4393-933b-85c84d3689eb None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:38.216837 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c3485-a099-4a49-a94f-d659570d5952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:38.217849 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c3485-a099-4a49-a94f-d659570d5952 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:38.218432 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c3485-a099-4a49-a94f-d659570d5952 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:38.218950 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c3485-a099-4a49-a94f-d659570d5952 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:38.220003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c3485-a099-4a49-a94f-d659570d5952 None admin] RBAC: Adding query filter params (role.id=4e60cc1f881a4caebdc208088b7ffcaa, user.id=952833e57cac4cf88e4d8e61c0f071e1, scope.system=all) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:38.220515 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c3485-a099-4a49-a94f-d659570d5952 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:38.222173 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c3485-a099-4a49-a94f-d659570d5952 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:38.227887 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 145/306] 10.222.0.15 () {56 vars in 1308 bytes} [Wed Jun 15 19:24:38 2022] GET /identity/v3/role_assignments?user.id=952833e57cac4cf88e4d8e61c0f071e1&scope.system=all&role.id=4e60cc1f881a4caebdc208088b7ffcaa => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:38.266825 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7258f3a3-4e30-424f-9725-ac332df84cd8 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:38.291758 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:38.313560 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b83954-d3a4-4281-a120-2e6363b2b90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:38.314675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b83954-d3a4-4281-a120-2e6363b2b90d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:38.315138 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b83954-d3a4-4281-a120-2e6363b2b90d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:38.315562 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b83954-d3a4-4281-a120-2e6363b2b90d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:38.325110 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16349e-76bd-4f74-bcc3-012f2285002d None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:38.325638 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16349e-76bd-4f74-bcc3-012f2285002d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:38.325889 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16349e-76bd-4f74-bcc3-012f2285002d None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:38.327781 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 146/307] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:38.331927 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b83954-d3a4-4281-a120-2e6363b2b90d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:38.333074 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11198454-5733-42f8-bfb7-d868b92599bb None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:38.333359 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11198454-5733-42f8-bfb7-d868b92599bb None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:38.333634 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11198454-5733-42f8-bfb7-d868b92599bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:38.341121 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-78b83954-d3a4-4281-a120-2e6363b2b90d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 19:24:38.341121 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:38.341121 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:38.341121 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:38.341121 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:38.341121 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:38.341121 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:38.341121 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:38.341121 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:38.341121 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:38.341121 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:38.341121 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:38.341121 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:38.341121 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:38.341121 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:38.341121 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:38.342063 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:38.342063 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:38.342063 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:38.342063 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:38.342063 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:38.342063 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:38.342063 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:38.342063 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:38.342063 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:38.342063 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:38.342063 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:38.342063 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:38.342063 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:38.342063 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:38.342063 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:38.342063 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:38.343117 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:38.343117 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:38.343117 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:38.343117 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:38.343117 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:38.343117 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:38.343117 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:38.343117 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:38.343117 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:38.343117 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:38.343117 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 19:24:38.343117 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:24:38.343117 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 162/308] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:24:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:24:38.348160 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b83954-d3a4-4281-a120-2e6363b2b90d None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:38.361613 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-11198454-5733-42f8-bfb7-d868b92599bb None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:38.363257 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-11198454-5733-42f8-bfb7-d868b92599bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:38.442850 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 147/309] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 19:24:38.464715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b83954-d3a4-4281-a120-2e6363b2b90d None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:38.464966 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11198454-5733-42f8-bfb7-d868b92599bb None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:38.500582 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaedf75-49b5-4f53-9df8-63f44a5f6acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:38.501363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaedf75-49b5-4f53-9df8-63f44a5f6acb None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:38.501734 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaedf75-49b5-4f53-9df8-63f44a5f6acb None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:38.502079 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaedf75-49b5-4f53-9df8-63f44a5f6acb None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:38.502932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaedf75-49b5-4f53-9df8-63f44a5f6acb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:38.503244 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaedf75-49b5-4f53-9df8-63f44a5f6acb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:38.504744 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaedf75-49b5-4f53-9df8-63f44a5f6acb None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:38.515063 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 163/310] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jun 15 19:24:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:24:38.520383 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaedf75-49b5-4f53-9df8-63f44a5f6acb None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:38.528575 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:38.567143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11198454-5733-42f8-bfb7-d868b92599bb None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:38.570571 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:38.602486 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f58faa5-5a38-49c4-861b-ef71c5e77d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:38.603284 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f58faa5-5a38-49c4-861b-ef71c5e77d08 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:38.603561 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f58faa5-5a38-49c4-861b-ef71c5e77d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:38.603834 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f58faa5-5a38-49c4-861b-ef71c5e77d08 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:38.615123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f58faa5-5a38-49c4-861b-ef71c5e77d08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:38.623024 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-9f58faa5-5a38-49c4-861b-ef71c5e77d08 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 19:24:38.623024 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:38.623024 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:38.623024 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:38.623024 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:38.623024 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:38.623024 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:38.623024 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:38.623024 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:38.623024 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:38.623024 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:38.623024 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:38.623024 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:38.623024 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:38.623024 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:38.623024 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:38.624096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:38.624096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:38.624096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:38.624096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:38.624096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:38.624096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:38.624096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:38.624096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:38.624096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:38.624096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:38.624096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:38.624096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:38.624096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:38.624096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:38.624096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:38.624096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:38.625087 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:38.625087 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:38.625087 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:38.625087 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:38.625087 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:38.625087 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:38.625087 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:38.625087 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:38.625087 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:38.625087 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:38.625087 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 19:24:38.625087 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:24:38.625087 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 148/311] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:24:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:24:38.632023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f58faa5-5a38-49c4-861b-ef71c5e77d08 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:38.636664 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaedf75-49b5-4f53-9df8-63f44a5f6acb None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:38.673277 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf23d3f-ed7f-40c9-903c-e771c8329feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:38.674463 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf23d3f-ed7f-40c9-903c-e771c8329feb None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:38.675037 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf23d3f-ed7f-40c9-903c-e771c8329feb None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:38.675499 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf23d3f-ed7f-40c9-903c-e771c8329feb None admin] PATH_INFO: `/v3/users/a574aa2e7fa843dda8e77b065bb25c71` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:38.676964 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf23d3f-ed7f-40c9-903c-e771c8329feb None admin] RBAC: Authorizing `identity:get_user(user_id=a574aa2e7fa843dda8e77b065bb25c71)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:38.678624 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf23d3f-ed7f-40c9-903c-e771c8329feb None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:38.680719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 164/312] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:24:38 2022] GET /identity/v3/users/a574aa2e7fa843dda8e77b065bb25c71 => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:24:38.686387 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf23d3f-ed7f-40c9-903c-e771c8329feb None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:38.739483 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f58faa5-5a38-49c4-861b-ef71c5e77d08 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:38.765979 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:38.779836 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092aae5-c274-48f4-96a1-cf6b26d0720e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:38.780207 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092aae5-c274-48f4-96a1-cf6b26d0720e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:38.780497 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092aae5-c274-48f4-96a1-cf6b26d0720e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:38.780834 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092aae5-c274-48f4-96a1-cf6b26d0720e None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:38.781678 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c092aae5-c274-48f4-96a1-cf6b26d0720e None admin] RBAC: Adding query filter params (name=member) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:38.781949 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c092aae5-c274-48f4-96a1-cf6b26d0720e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:38.783487 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c092aae5-c274-48f4-96a1-cf6b26d0720e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:38.792772 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 149/313] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jun 15 19:24:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:38.793374 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf23d3f-ed7f-40c9-903c-e771c8329feb None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:38.798157 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092aae5-c274-48f4-96a1-cf6b26d0720e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:38.804749 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:38.830617 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd39206e-5cd3-4234-bebc-7d556985d712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:38.831375 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39206e-5cd3-4234-bebc-7d556985d712 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:38.831722 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39206e-5cd3-4234-bebc-7d556985d712 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:38.832055 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39206e-5cd3-4234-bebc-7d556985d712 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:38.901012 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd39206e-5cd3-4234-bebc-7d556985d712 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:38.902817 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd39206e-5cd3-4234-bebc-7d556985d712 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:38.914250 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092aae5-c274-48f4-96a1-cf6b26d0720e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:38.951871 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3d7ad-fc1b-4bde-b0d1-2dacfb20d0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:38.953100 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3d7ad-fc1b-4bde-b0d1-2dacfb20d0a2 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:38.953554 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3d7ad-fc1b-4bde-b0d1-2dacfb20d0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:38.953973 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3d7ad-fc1b-4bde-b0d1-2dacfb20d0a2 None admin] PATH_INFO: `/v3/users/bc387c006e2047878b5ccb05a452a2c6` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:38.955525 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee3d7ad-fc1b-4bde-b0d1-2dacfb20d0a2 None admin] RBAC: Authorizing `identity:get_user(user_id=bc387c006e2047878b5ccb05a452a2c6)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:38.957201 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee3d7ad-fc1b-4bde-b0d1-2dacfb20d0a2 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:38.964080 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 150/314] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:24:38 2022] GET /identity/v3/users/bc387c006e2047878b5ccb05a452a2c6 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:24:38.965703 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3d7ad-fc1b-4bde-b0d1-2dacfb20d0a2 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:39.009697 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 165/315] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:24:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 19:24:39.010818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd39206e-5cd3-4234-bebc-7d556985d712 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:39.022034 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:39.028725 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:39.080685 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3d7ad-fc1b-4bde-b0d1-2dacfb20d0a2 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:39.116959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd39206e-5cd3-4234-bebc-7d556985d712 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:39.117463 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f628bca-9434-4844-a3bc-3584520d9c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:39.118646 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f628bca-9434-4844-a3bc-3584520d9c61 None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:39.119210 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f628bca-9434-4844-a3bc-3584520d9c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:39.119666 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f628bca-9434-4844-a3bc-3584520d9c61 None admin] PATH_INFO: `/v3/system/users/bc387c006e2047878b5ccb05a452a2c6/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:39.121492 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f628bca-9434-4844-a3bc-3584520d9c61 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=bc387c006e2047878b5ccb05a452a2c6, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:39.123163 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f628bca-9434-4844-a3bc-3584520d9c61 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:39.142364 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 151/316] 10.222.0.15 () {58 vars in 1334 bytes} [Wed Jun 15 19:24:38 2022] PUT /identity/v3/system/users/bc387c006e2047878b5ccb05a452a2c6/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:24:39.162992 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955fb6b9-364d-44f2-a0bf-6c61765a633e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:39.163937 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955fb6b9-364d-44f2-a0bf-6c61765a633e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:39.164221 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955fb6b9-364d-44f2-a0bf-6c61765a633e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:39.164493 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955fb6b9-364d-44f2-a0bf-6c61765a633e None admin] PATH_INFO: `/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:39.165760 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955fb6b9-364d-44f2-a0bf-6c61765a633e None admin] RBAC: Authorizing `identity:get_project(project_id=383a1a7fd67a4b0c8baf1403a8b02f25)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:39.167343 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955fb6b9-364d-44f2-a0bf-6c61765a633e None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:39.169662 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 166/317] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:39 2022] GET /identity/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:24:39.176171 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f628bca-9434-4844-a3bc-3584520d9c61 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:39.296944 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f628bca-9434-4844-a3bc-3584520d9c61 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:39.316245 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:39.351852 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8708c16-20f0-46ab-b230-cb48ee6a01b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:39.353354 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8708c16-20f0-46ab-b230-cb48ee6a01b2 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:39.353874 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8708c16-20f0-46ab-b230-cb48ee6a01b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:39.354383 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8708c16-20f0-46ab-b230-cb48ee6a01b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:39.355504 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8708c16-20f0-46ab-b230-cb48ee6a01b2 None admin] RBAC: Adding query filter params (role.id=4e60cc1f881a4caebdc208088b7ffcaa, scope.project.id=383a1a7fd67a4b0c8baf1403a8b02f25, user.id=a574aa2e7fa843dda8e77b065bb25c71) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:39.356056 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8708c16-20f0-46ab-b230-cb48ee6a01b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:39.357608 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8708c16-20f0-46ab-b230-cb48ee6a01b2 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:39.364695 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 152/318] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:24:39 2022] GET /identity/v3/role_assignments?user.id=a574aa2e7fa843dda8e77b065bb25c71&scope.project.id=383a1a7fd67a4b0c8baf1403a8b02f25&role.id=4e60cc1f881a4caebdc208088b7ffcaa => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:40.174831 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714569a-7e78-431a-8534-f94ee554691f None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:40.175384 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714569a-7e78-431a-8534-f94ee554691f None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:40.175384 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714569a-7e78-431a-8534-f94ee554691f None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:40.176777 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 167/319] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:40.181826 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e781c2-0238-4e0d-b8bb-90608058d0da None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:40.182147 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e781c2-0238-4e0d-b8bb-90608058d0da None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:40.182429 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e781c2-0238-4e0d-b8bb-90608058d0da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:40.199076 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-f9e781c2-0238-4e0d-b8bb-90608058d0da None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:40.200794 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f9e781c2-0238-4e0d-b8bb-90608058d0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:40.270621 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 153/320] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:40.292956 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5714569a-7e78-431a-8534-f94ee554691f None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:40.383173 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:40.407527 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df642e3-0aa2-477c-b881-ff607ea55c9c None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:40.408054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5714569a-7e78-431a-8534-f94ee554691f None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:40.408136 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df642e3-0aa2-477c-b881-ff607ea55c9c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:40.408136 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df642e3-0aa2-477c-b881-ff607ea55c9c None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:40.409604 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 154/321] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 19:24:40.414263 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7938db-06fd-4eea-afdf-9e879f0c9692 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:40.414553 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7938db-06fd-4eea-afdf-9e879f0c9692 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:40.414827 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7938db-06fd-4eea-afdf-9e879f0c9692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:40.431556 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-0d7938db-06fd-4eea-afdf-9e879f0c9692 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:40.432654 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0d7938db-06fd-4eea-afdf-9e879f0c9692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:40.444932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5805e60d-e9da-48c7-a47b-cfc4cc189ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:40.445727 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805e60d-e9da-48c7-a47b-cfc4cc189ceb None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:40.446027 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805e60d-e9da-48c7-a47b-cfc4cc189ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:40.446322 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805e60d-e9da-48c7-a47b-cfc4cc189ceb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:40.447484 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5805e60d-e9da-48c7-a47b-cfc4cc189ceb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:40.448912 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5805e60d-e9da-48c7-a47b-cfc4cc189ceb None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:40.455017 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 168/322] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:24:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:40.456609 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5805e60d-e9da-48c7-a47b-cfc4cc189ceb None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:40.508075 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 155/323] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 19:24:40.530039 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7938db-06fd-4eea-afdf-9e879f0c9692 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:40.561333 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5805e60d-e9da-48c7-a47b-cfc4cc189ceb None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:40.570887 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:40.596619 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a7fe1b-89d3-41ba-8bb8-259e133c1fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:40.597276 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a7fe1b-89d3-41ba-8bb8-259e133c1fd1 None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:40.597557 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a7fe1b-89d3-41ba-8bb8-259e133c1fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:40.597830 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a7fe1b-89d3-41ba-8bb8-259e133c1fd1 None admin] PATH_INFO: `/v3/groups` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:40.598550 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a7fe1b-89d3-41ba-8bb8-259e133c1fd1 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:40.599965 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a7fe1b-89d3-41ba-8bb8-259e133c1fd1 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:40.626492 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 169/324] 10.222.0.15 () {60 vars in 1134 bytes} [Wed Jun 15 19:24:40 2022] POST /identity/v3/groups => generated 226 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:24:40.629849 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7938db-06fd-4eea-afdf-9e879f0c9692 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:40.667340 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf307a36-5498-471c-a5fb-7f85e966be42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:40.668704 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf307a36-5498-471c-a5fb-7f85e966be42 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:40.668982 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf307a36-5498-471c-a5fb-7f85e966be42 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:40.669253 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf307a36-5498-471c-a5fb-7f85e966be42 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:40.681086 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf307a36-5498-471c-a5fb-7f85e966be42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:40.689934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-bf307a36-5498-471c-a5fb-7f85e966be42 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 19:24:40.689934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:40.689934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:40.689934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:40.689934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:40.689934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:40.689934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:40.689934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:40.689934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:40.689934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:40.689934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:40.689934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:40.689934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:40.689934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:40.689934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:40.689934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:40.691058 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:40.691058 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:40.691058 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:40.691058 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:40.691058 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:40.691058 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:40.691058 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:40.691058 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:40.691058 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:40.691058 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:40.691058 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:40.691058 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:40.691058 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:40.691058 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:40.691058 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:40.691058 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:40.691861 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:40.691861 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:40.691861 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:40.691861 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:40.691861 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:40.691861 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:40.691861 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:40.691861 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:40.691861 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:40.691861 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:40.691861 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 19:24:40.691861 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:24:40.694438 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 156/325] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:24:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:24:40.697132 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a7fe1b-89d3-41ba-8bb8-259e133c1fd1 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:40.706691 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:40.805270 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a7fe1b-89d3-41ba-8bb8-259e133c1fd1 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:40.839678 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5e071c-50d6-41d6-8895-288de7cdbd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:40.840378 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e071c-50d6-41d6-8895-288de7cdbd32 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:40.840658 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e071c-50d6-41d6-8895-288de7cdbd32 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:40.840938 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e071c-50d6-41d6-8895-288de7cdbd32 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:40.841685 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5e071c-50d6-41d6-8895-288de7cdbd32 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:40.841988 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5e071c-50d6-41d6-8895-288de7cdbd32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:40.843385 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5e071c-50d6-41d6-8895-288de7cdbd32 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:40.851830 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 170/326] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jun 15 19:24:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:40.857880 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf307a36-5498-471c-a5fb-7f85e966be42 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:40.969157 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf307a36-5498-471c-a5fb-7f85e966be42 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:40.971183 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:41.005977 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfffbf3-8d22-441a-8e3d-7aaed0f299d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:41.006939 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfffbf3-8d22-441a-8e3d-7aaed0f299d1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:41.007482 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfffbf3-8d22-441a-8e3d-7aaed0f299d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:41.007482 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfffbf3-8d22-441a-8e3d-7aaed0f299d1 None admin] PATH_INFO: `/v3/users/952833e57cac4cf88e4d8e61c0f071e1` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:41.009033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfffbf3-8d22-441a-8e3d-7aaed0f299d1 None admin] RBAC: Authorizing `identity:get_user(user_id=952833e57cac4cf88e4d8e61c0f071e1)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:41.010528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfffbf3-8d22-441a-8e3d-7aaed0f299d1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:41.012606 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 157/327] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:24:40 2022] GET /identity/v3/users/952833e57cac4cf88e4d8e61c0f071e1 => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:41.018019 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5e071c-50d6-41d6-8895-288de7cdbd32 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:41.093139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:41.129245 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5e071c-50d6-41d6-8895-288de7cdbd32 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:41.170260 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9354ff84-d523-4935-8cb6-6b8b667e6bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:41.171141 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354ff84-d523-4935-8cb6-6b8b667e6bec None admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:41.171422 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354ff84-d523-4935-8cb6-6b8b667e6bec None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:41.171698 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354ff84-d523-4935-8cb6-6b8b667e6bec None admin] PATH_INFO: `/v3/system/users/952833e57cac4cf88e4d8e61c0f071e1/roles/4e60cc1f881a4caebdc208088b7ffcaa` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:41.174560 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9354ff84-d523-4935-8cb6-6b8b667e6bec None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=952833e57cac4cf88e4d8e61c0f071e1, role_id=4e60cc1f881a4caebdc208088b7ffcaa)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:41.175943 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9354ff84-d523-4935-8cb6-6b8b667e6bec None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:41.193069 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 171/328] 10.222.0.15 () {58 vars in 1334 bytes} [Wed Jun 15 19:24:41 2022] PUT /identity/v3/system/users/952833e57cac4cf88e4d8e61c0f071e1/roles/4e60cc1f881a4caebdc208088b7ffcaa => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:24:41.342732 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5d543-fa19-4694-939a-33e0f7348a7c None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:41.343094 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5d543-fa19-4694-939a-33e0f7348a7c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:41.343382 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5d543-fa19-4694-939a-33e0f7348a7c None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:41.345151 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 158/329] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:41.350422 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87021212-f1de-4a73-94f2-85007bc424b0 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:41.350713 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87021212-f1de-4a73-94f2-85007bc424b0 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:41.350997 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87021212-f1de-4a73-94f2-85007bc424b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:41.370339 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-87021212-f1de-4a73-94f2-85007bc424b0 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:41.371615 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-87021212-f1de-4a73-94f2-85007bc424b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:41.450529 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 172/330] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:41.474654 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a5d543-fa19-4694-939a-33e0f7348a7c None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:41.596605 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a5d543-fa19-4694-939a-33e0f7348a7c None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:41.635729 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63428f73-b16b-42a5-93de-5d2a7e603b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:41.636534 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63428f73-b16b-42a5-93de-5d2a7e603b29 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:41.637353 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63428f73-b16b-42a5-93de-5d2a7e603b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:41.637630 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63428f73-b16b-42a5-93de-5d2a7e603b29 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:41.653726 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63428f73-b16b-42a5-93de-5d2a7e603b29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:41.661649 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-63428f73-b16b-42a5-93de-5d2a7e603b29 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 19:24:41.661649 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:41.661649 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:41.661649 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:41.661649 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:41.661649 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:41.661649 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:41.661649 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:41.661649 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:41.661649 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:41.661649 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:41.661649 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:41.661649 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:41.661649 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:41.661649 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:41.661649 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:41.662585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:41.662585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:41.662585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:41.662585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:41.662585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:41.662585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:41.662585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:41.662585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:41.662585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:41.662585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:41.662585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:41.662585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:41.662585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:41.662585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:41.662585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:41.662585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:41.663371 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:41.663371 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:41.663371 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:41.663371 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:41.663371 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:41.663371 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:41.663371 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:41.663371 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:41.663371 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:41.663371 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:41.663371 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 19:24:41.663371 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:24:41.663371 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 159/331] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:24:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:41.669015 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87021212-f1de-4a73-94f2-85007bc424b0 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:41.747757 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:41.794217 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87021212-f1de-4a73-94f2-85007bc424b0 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:41.837132 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f711e32-5b06-479a-97b7-cbd23e392324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:41.837815 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f711e32-5b06-479a-97b7-cbd23e392324 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:41.838116 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f711e32-5b06-479a-97b7-cbd23e392324 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:41.838399 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f711e32-5b06-479a-97b7-cbd23e392324 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:41.839464 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f711e32-5b06-479a-97b7-cbd23e392324 None admin] RBAC: Adding query filter params (name=member) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:41.840067 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f711e32-5b06-479a-97b7-cbd23e392324 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:41.841693 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f711e32-5b06-479a-97b7-cbd23e392324 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:41.857715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 173/332] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jun 15 19:24:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:41.863002 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63428f73-b16b-42a5-93de-5d2a7e603b29 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:41.908396 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:42.001278 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63428f73-b16b-42a5-93de-5d2a7e603b29 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:42.043956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b6218-7445-4e1b-b3e0-321d353fdee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:42.045203 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b6218-7445-4e1b-b3e0-321d353fdee9 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:42.045686 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b6218-7445-4e1b-b3e0-321d353fdee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:42.046189 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b6218-7445-4e1b-b3e0-321d353fdee9 None admin] PATH_INFO: `/v3/users/bc387c006e2047878b5ccb05a452a2c6` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:42.047759 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b6218-7445-4e1b-b3e0-321d353fdee9 None admin] RBAC: Authorizing `identity:get_user(user_id=bc387c006e2047878b5ccb05a452a2c6)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:42.049445 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b6218-7445-4e1b-b3e0-321d353fdee9 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:42.051373 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 160/333] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:24:41 2022] GET /identity/v3/users/bc387c006e2047878b5ccb05a452a2c6 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:42.057117 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f711e32-5b06-479a-97b7-cbd23e392324 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:42.190071 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f711e32-5b06-479a-97b7-cbd23e392324 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:42.218991 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:42.231671 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d571de0b-284f-4e11-a5e9-21d7428c74e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:42.232524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571de0b-284f-4e11-a5e9-21d7428c74e1 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:42.233033 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571de0b-284f-4e11-a5e9-21d7428c74e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:42.233311 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571de0b-284f-4e11-a5e9-21d7428c74e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:42.234180 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d571de0b-284f-4e11-a5e9-21d7428c74e1 None admin] RBAC: Adding query filter params (role.id=e53a78912e764f8387e0cf637bb86bee, user.id=bc387c006e2047878b5ccb05a452a2c6, scope.system=all) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:42.234493 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d571de0b-284f-4e11-a5e9-21d7428c74e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:42.235946 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d571de0b-284f-4e11-a5e9-21d7428c74e1 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:42.254742 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 174/334] 10.222.0.15 () {56 vars in 1308 bytes} [Wed Jun 15 19:24:42 2022] GET /identity/v3/role_assignments?user.id=bc387c006e2047878b5ccb05a452a2c6&scope.system=all&role.id=e53a78912e764f8387e0cf637bb86bee => generated 510 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:43.555510 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1785c-1311-42bc-a1ac-cf80d93c124e None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:43.556546 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1785c-1311-42bc-a1ac-cf80d93c124e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:43.557032 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1785c-1311-42bc-a1ac-cf80d93c124e None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:43.559486 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 161/335] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:43.564599 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9577a-198c-4fef-931b-fef111222e66 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:43.565046 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9577a-198c-4fef-931b-fef111222e66 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:43.565135 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9577a-198c-4fef-931b-fef111222e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:43.588412 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-f0b9577a-198c-4fef-931b-fef111222e66 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:43.589969 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-f0b9577a-198c-4fef-931b-fef111222e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:43.669161 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 175/336] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 19:24:43.707082 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1785c-1311-42bc-a1ac-cf80d93c124e None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:43.831972 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:43.838960 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1785c-1311-42bc-a1ac-cf80d93c124e None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:43.878714 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca41cbd-1de4-4126-992b-e069cb20c572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:43.879514 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca41cbd-1de4-4126-992b-e069cb20c572 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:43.879795 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca41cbd-1de4-4126-992b-e069cb20c572 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:43.880070 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca41cbd-1de4-4126-992b-e069cb20c572 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:43.895385 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca41cbd-1de4-4126-992b-e069cb20c572 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:43.903204 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-2ca41cbd-1de4-4126-992b-e069cb20c572 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 19:24:43.903204 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:43.903204 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:43.903204 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:43.903204 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:43.903204 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:43.903204 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:43.903204 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:43.903204 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:43.903204 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:43.903204 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:43.903204 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:43.903204 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:43.903204 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:43.903204 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:43.903204 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:43.904080 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:43.904080 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:43.904080 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:43.904080 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:43.904080 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:43.904080 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:43.904080 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:43.904080 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:43.904080 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:43.904080 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:43.904080 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:43.904080 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:43.904080 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:43.904080 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:43.904080 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:43.904080 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:43.904867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:43.904867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:43.904867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:43.904867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:43.904867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:43.904867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:43.904867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:43.904867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:43.904867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:43.904867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:43.904867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 19:24:43.904867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:24:43.904867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 162/337] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:24:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:43.911378 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b9577a-198c-4fef-931b-fef111222e66 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:44.028315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b9577a-198c-4fef-931b-fef111222e66 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:44.038838 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86249126-570d-455c-9648-c507b0c09002 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:44.039123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86249126-570d-455c-9648-c507b0c09002 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:44.039397 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86249126-570d-455c-9648-c507b0c09002 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:44.040781 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 163/338] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:44.047439 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:44.048725 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9179b-d3e6-48f9-811c-29aa9993bbcc None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:44.049015 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9179b-d3e6-48f9-811c-29aa9993bbcc None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:44.049290 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9179b-d3e6-48f9-811c-29aa9993bbcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:44.073742 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-50e9179b-d3e6-48f9-811c-29aa9993bbcc None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:44.076025 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-50e9179b-d3e6-48f9-811c-29aa9993bbcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:44.082427 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f91fe4-95a1-4459-a157-0ede31bc79a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:44.083109 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f91fe4-95a1-4459-a157-0ede31bc79a5 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:44.083395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f91fe4-95a1-4459-a157-0ede31bc79a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:44.083675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f91fe4-95a1-4459-a157-0ede31bc79a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:44.084920 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f91fe4-95a1-4459-a157-0ede31bc79a5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:44.085240 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f91fe4-95a1-4459-a157-0ede31bc79a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:44.087048 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f91fe4-95a1-4459-a157-0ede31bc79a5 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:44.100131 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 176/339] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jun 15 19:24:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:44.106998 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f91fe4-95a1-4459-a157-0ede31bc79a5 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:44.200136 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 164/340] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 19:24:44.266142 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f91fe4-95a1-4459-a157-0ede31bc79a5 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:44.294985 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9179b-d3e6-48f9-811c-29aa9993bbcc None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:44.304175 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc43e70-f35e-4eca-989f-fb74ff231da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:44.304827 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc43e70-f35e-4eca-989f-fb74ff231da4 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:44.305107 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc43e70-f35e-4eca-989f-fb74ff231da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:44.305385 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc43e70-f35e-4eca-989f-fb74ff231da4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:44.364635 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:44.393736 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc43e70-f35e-4eca-989f-fb74ff231da4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:44.395249 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc43e70-f35e-4eca-989f-fb74ff231da4 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:44.504113 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 177/341] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:24:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 19:24:44.514112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc43e70-f35e-4eca-989f-fb74ff231da4 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:44.564504 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:44.587418 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9179b-d3e6-48f9-811c-29aa9993bbcc None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:44.645893 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f0640-7630-4995-9714-ca5388a2e36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:44.646737 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f0640-7630-4995-9714-ca5388a2e36c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:44.647016 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f0640-7630-4995-9714-ca5388a2e36c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:44.647284 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f0640-7630-4995-9714-ca5388a2e36c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:44.664036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:44.669488 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66f0640-7630-4995-9714-ca5388a2e36c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:44.680135 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-f66f0640-7630-4995-9714-ca5388a2e36c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 19:24:44.680135 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:44.680135 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:44.680135 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:44.680135 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:44.680135 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:44.680135 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:44.680135 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:44.680135 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:44.680135 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:44.680135 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:44.680135 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:44.680135 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:44.680135 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:44.680135 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:44.680135 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:44.681292 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:44.681292 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:44.681292 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:44.681292 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:44.681292 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:44.681292 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:44.681292 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:44.681292 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:44.681292 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:44.681292 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:44.681292 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:44.681292 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:44.681292 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:44.681292 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:44.681292 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:44.681292 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:44.682241 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:44.682241 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:44.682241 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:44.682241 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:44.682241 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:44.682241 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:44.682241 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:44.682241 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:44.682241 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:44.682241 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:44.682241 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 19:24:44.682241 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:24:44.682241 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 165/342] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jun 15 19:24:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:44.691221 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f0640-7630-4995-9714-ca5388a2e36c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:44.722886 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc43e70-f35e-4eca-989f-fb74ff231da4 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:44.763355 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914a5c7f-8683-4ea1-a71b-dd9ac1921b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:44.764547 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a5c7f-8683-4ea1-a71b-dd9ac1921b60 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:44.765007 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a5c7f-8683-4ea1-a71b-dd9ac1921b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:44.765455 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a5c7f-8683-4ea1-a71b-dd9ac1921b60 None admin] PATH_INFO: `/v3/projects/c4f989e5fad14bcebae5e279f22ee21d` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:44.767052 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914a5c7f-8683-4ea1-a71b-dd9ac1921b60 None admin] RBAC: Authorizing `identity:get_project(project_id=c4f989e5fad14bcebae5e279f22ee21d)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:44.768816 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914a5c7f-8683-4ea1-a71b-dd9ac1921b60 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:44.771242 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 178/343] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:44 2022] GET /identity/v3/projects/c4f989e5fad14bcebae5e279f22ee21d => generated 298 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:44.772780 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c156402-070e-4317-9296-387c22805798 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:44.773242 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c156402-070e-4317-9296-387c22805798 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:44.773690 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c156402-070e-4317-9296-387c22805798 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:44.775374 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 179/344] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:44.781038 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef24b236-5f6e-4c07-83fe-d05a0d16dfcd None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:44.781528 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef24b236-5f6e-4c07-83fe-d05a0d16dfcd None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:44.781978 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef24b236-5f6e-4c07-83fe-d05a0d16dfcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:44.802685 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-ef24b236-5f6e-4c07-83fe-d05a0d16dfcd None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:44.804239 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-ef24b236-5f6e-4c07-83fe-d05a0d16dfcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:44.818189 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f0640-7630-4995-9714-ca5388a2e36c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:44.857622 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea8f08-b78e-4457-b6bb-8c4512d95925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:44.861532 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea8f08-b78e-4457-b6bb-8c4512d95925 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:44.861532 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea8f08-b78e-4457-b6bb-8c4512d95925 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:44.861532 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea8f08-b78e-4457-b6bb-8c4512d95925 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:44.861532 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea8f08-b78e-4457-b6bb-8c4512d95925 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:44.861532 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea8f08-b78e-4457-b6bb-8c4512d95925 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:44.861532 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea8f08-b78e-4457-b6bb-8c4512d95925 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:44.874808 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 166/345] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:24:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:44.876162 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05e18e-71c5-4c80-8158-d9236b9e5463 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:44.876454 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05e18e-71c5-4c80-8158-d9236b9e5463 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:44.876730 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05e18e-71c5-4c80-8158-d9236b9e5463 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:44.878284 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 167/346] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:44.878998 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b05e18e-71c5-4c80-8158-d9236b9e5463 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:44.893113 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:44.894321 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 180/347] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:44.895238 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef24b236-5f6e-4c07-83fe-d05a0d16dfcd None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:45.002534 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:45.012008 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b05e18e-71c5-4c80-8158-d9236b9e5463 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:45.040132 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef24b236-5f6e-4c07-83fe-d05a0d16dfcd None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:45.053883 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab3fcdc-6d08-46b5-9e2c-94d8472054d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:45.054826 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3fcdc-6d08-46b5-9e2c-94d8472054d8 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:45.055106 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3fcdc-6d08-46b5-9e2c-94d8472054d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:45.055541 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3fcdc-6d08-46b5-9e2c-94d8472054d8 None admin] PATH_INFO: `/v3/groups/4fd7c2055c3147f5ad7054d54f3a1ad2` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:45.062698 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab3fcdc-6d08-46b5-9e2c-94d8472054d8 None admin] RBAC: Authorizing `identity:get_group(group_id=4fd7c2055c3147f5ad7054d54f3a1ad2)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:45.064323 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab3fcdc-6d08-46b5-9e2c-94d8472054d8 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:45.066006 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 168/348] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Jun 15 19:24:44 2022] GET /identity/v3/groups/4fd7c2055c3147f5ad7054d54f3a1ad2 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:45.067265 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20b057-2f84-4138-807d-ddd9f5a9d7c5 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:45.067551 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20b057-2f84-4138-807d-ddd9f5a9d7c5 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:45.067979 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20b057-2f84-4138-807d-ddd9f5a9d7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:45.084502 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b411bdcd-d257-48d8-bbf7-d3beadbe319f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:45.085246 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411bdcd-d257-48d8-bbf7-d3beadbe319f None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:45.085537 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411bdcd-d257-48d8-bbf7-d3beadbe319f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:45.085814 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411bdcd-d257-48d8-bbf7-d3beadbe319f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:45.087293 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-9c20b057-2f84-4138-807d-ddd9f5a9d7c5 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:45.088488 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-9c20b057-2f84-4138-807d-ddd9f5a9d7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:45.180049 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 169/349] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:45.180049 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20b057-2f84-4138-807d-ddd9f5a9d7c5 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:45.181502 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b411bdcd-d257-48d8-bbf7-d3beadbe319f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:45.183058 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b411bdcd-d257-48d8-bbf7-d3beadbe319f None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:45.183530 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:45.270984 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:45.317775 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 181/350] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:24:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 19:24:45.322653 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20b057-2f84-4138-807d-ddd9f5a9d7c5 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:45.322787 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ec31c-e62e-434e-b128-167f860938f8 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:45.322787 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ec31c-e62e-434e-b128-167f860938f8 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:45.322787 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ec31c-e62e-434e-b128-167f860938f8 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:45.322787 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 182/351] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:45.323053 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ec31c-e62e-434e-b128-167f860938f8 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:45.371949 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573dc18-4e03-4896-a420-2a1863e566d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:45.372746 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573dc18-4e03-4896-a420-2a1863e566d9 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:45.373023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573dc18-4e03-4896-a420-2a1863e566d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:45.373479 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573dc18-4e03-4896-a420-2a1863e566d9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:45.392510 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0573dc18-4e03-4896-a420-2a1863e566d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:45.400291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-0573dc18-4e03-4896-a420-2a1863e566d9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 19:24:45.400291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:45.400291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:45.400291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:45.400291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:45.400291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:45.400291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:45.400291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:45.400291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:45.400291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:45.400291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:45.400291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:45.400291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:45.400291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:45.400291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:45.400291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:45.401228 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:45.401228 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:45.401228 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:45.401228 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:45.401228 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:45.401228 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:45.401228 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:45.401228 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:45.401228 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:45.401228 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:45.401228 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:45.401228 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:45.401228 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:45.401228 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:45.401228 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:45.401228 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:45.402047 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:45.402047 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:45.402047 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:45.402047 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:45.402047 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:45.402047 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:45.402047 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:45.402047 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:45.402047 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:45.402047 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:45.402047 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 19:24:45.402047 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:24:45.402047 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 170/352] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:24:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:45.402793 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573dc18-4e03-4896-a420-2a1863e566d9 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:45.477731 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ec31c-e62e-434e-b128-167f860938f8 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:45.486524 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:45.516560 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6383e70d-ebc2-41cd-99a2-8962f4dafdbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:45.517215 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383e70d-ebc2-41cd-99a2-8962f4dafdbe None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:45.517496 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383e70d-ebc2-41cd-99a2-8962f4dafdbe None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:45.517770 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383e70d-ebc2-41cd-99a2-8962f4dafdbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:45.518644 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6383e70d-ebc2-41cd-99a2-8962f4dafdbe None admin] RBAC: Adding query filter params (group.id=4fd7c2055c3147f5ad7054d54f3a1ad2, role.id=e53a78912e764f8387e0cf637bb86bee, scope.project.id=c4f989e5fad14bcebae5e279f22ee21d) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:45.518956 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6383e70d-ebc2-41cd-99a2-8962f4dafdbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:45.520397 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6383e70d-ebc2-41cd-99a2-8962f4dafdbe None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:45.526467 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 183/353] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Jun 15 19:24:45 2022] GET /identity/v3/role_assignments?group.id=4fd7c2055c3147f5ad7054d54f3a1ad2&scope.project.id=c4f989e5fad14bcebae5e279f22ee21d&role.id=e53a78912e764f8387e0cf637bb86bee => generated 260 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:45.534418 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8f5ad-8d00-4c85-9421-112e8b208ee6 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:45.534713 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8f5ad-8d00-4c85-9421-112e8b208ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:45.534993 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8f5ad-8d00-4c85-9421-112e8b208ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:45.538015 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573dc18-4e03-4896-a420-2a1863e566d9 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:45.561438 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-dda8f5ad-8d00-4c85-9421-112e8b208ee6 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:45.563686 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-dda8f5ad-8d00-4c85-9421-112e8b208ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:45.590974 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d00159-800f-4ccf-a0b1-589d56066e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:45.593333 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d00159-800f-4ccf-a0b1-589d56066e46 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:45.594165 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d00159-800f-4ccf-a0b1-589d56066e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:45.594654 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d00159-800f-4ccf-a0b1-589d56066e46 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:45.599092 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:45.615929 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d00159-800f-4ccf-a0b1-589d56066e46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:45.629027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-f9d00159-800f-4ccf-a0b1-589d56066e46 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 19:24:45.629027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:45.629027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:45.629027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:45.629027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:45.629027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:45.629027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:45.629027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:45.629027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:45.629027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:45.629027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:45.629027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:45.629027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:45.629027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:45.629027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:45.629027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:45.629997 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:45.629997 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:45.629997 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:45.629997 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:45.629997 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:45.629997 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:45.629997 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:45.629997 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:45.629997 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:45.629997 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:45.629997 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:45.629997 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:45.629997 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:45.629997 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:45.629997 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:45.629997 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:45.630830 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:45.630830 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:45.630830 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:45.630830 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:45.630830 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:45.630830 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:45.630830 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:45.630830 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:45.630830 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:45.630830 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:45.630830 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 19:24:45.630830 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:24:45.634616 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 171/354] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jun 15 19:24:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:45.635870 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d00159-800f-4ccf-a0b1-589d56066e46 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:45.666246 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 184/355] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:45.667258 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8f5ad-8d00-4c85-9421-112e8b208ee6 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:45.779379 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d00159-800f-4ccf-a0b1-589d56066e46 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:45.802035 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8f5ad-8d00-4c85-9421-112e8b208ee6 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:45.811534 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:45.827394 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8725ac-b6a1-466f-959f-6ba84b76a031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:45.828270 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8725ac-b6a1-466f-959f-6ba84b76a031 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:45.828826 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8725ac-b6a1-466f-959f-6ba84b76a031 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:45.829103 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8725ac-b6a1-466f-959f-6ba84b76a031 None admin] PATH_INFO: `/v3/projects/c4f989e5fad14bcebae5e279f22ee21d` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:45.830951 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8725ac-b6a1-466f-959f-6ba84b76a031 None admin] RBAC: Authorizing `identity:get_project(project_id=c4f989e5fad14bcebae5e279f22ee21d)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:45.832418 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8725ac-b6a1-466f-959f-6ba84b76a031 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:45.834311 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 172/356] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:45 2022] GET /identity/v3/projects/c4f989e5fad14bcebae5e279f22ee21d => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:45.834984 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8725ac-b6a1-466f-959f-6ba84b76a031 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:45.844004 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d7314-d3ae-4c2a-8672-40032af8a365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:45.845018 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d7314-d3ae-4c2a-8672-40032af8a365 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:45.845018 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d7314-d3ae-4c2a-8672-40032af8a365 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:45.845207 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d7314-d3ae-4c2a-8672-40032af8a365 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:45.846068 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d7314-d3ae-4c2a-8672-40032af8a365 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:45.846422 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d7314-d3ae-4c2a-8672-40032af8a365 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:45.847934 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d7314-d3ae-4c2a-8672-40032af8a365 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:45.858263 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 185/357] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jun 15 19:24:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:45.859642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33128ed9-e7b9-4ede-9b8a-46c88429f2cb None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:45.859923 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33128ed9-e7b9-4ede-9b8a-46c88429f2cb None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:45.860204 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33128ed9-e7b9-4ede-9b8a-46c88429f2cb None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:45.861494 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 186/358] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:45.862219 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33128ed9-e7b9-4ede-9b8a-46c88429f2cb None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:45.903041 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:45.968889 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8725ac-b6a1-466f-959f-6ba84b76a031 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:46.005582 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33128ed9-e7b9-4ede-9b8a-46c88429f2cb None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:46.011001 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c0710-2804-47be-8f60-7ce62c6ed502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:46.013624 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c0710-2804-47be-8f60-7ce62c6ed502 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:46.013624 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c0710-2804-47be-8f60-7ce62c6ed502 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:46.013624 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c0710-2804-47be-8f60-7ce62c6ed502 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:46.013624 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c0710-2804-47be-8f60-7ce62c6ed502 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:46.013942 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c0710-2804-47be-8f60-7ce62c6ed502 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:46.015732 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c0710-2804-47be-8f60-7ce62c6ed502 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:46.027330 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 173/359] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:24:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:46.028276 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c0710-2804-47be-8f60-7ce62c6ed502 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:46.057089 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1225c0-5114-4359-af77-4aeafd8500a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:46.058463 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1225c0-5114-4359-af77-4aeafd8500a6 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:46.059883 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1225c0-5114-4359-af77-4aeafd8500a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:46.060398 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1225c0-5114-4359-af77-4aeafd8500a6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:46.074460 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1225c0-5114-4359-af77-4aeafd8500a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:46.082885 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-4f1225c0-5114-4359-af77-4aeafd8500a6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 19:24:46.082885 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:46.082885 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:46.082885 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:46.082885 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:46.082885 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:46.082885 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:46.082885 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:46.082885 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:46.082885 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:46.082885 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:46.082885 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:46.082885 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:46.082885 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:46.082885 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:46.082885 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:46.084031 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:46.084031 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:46.084031 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:46.084031 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:46.084031 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:46.084031 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:46.084031 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:46.084031 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:46.084031 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:46.084031 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:46.084031 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:46.084031 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:46.084031 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:46.084031 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:46.084031 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:46.084031 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:46.085042 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:46.085042 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:46.085042 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:46.085042 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:46.085042 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:46.085042 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:46.085042 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:46.085042 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:46.085042 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:46.085042 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:46.085042 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 19:24:46.085042 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:24:46.087296 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 187/360] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:24:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:46.088368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1225c0-5114-4359-af77-4aeafd8500a6 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:46.103214 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:46.164559 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c0710-2804-47be-8f60-7ce62c6ed502 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:46.192962 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:46.205636 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc51e6d-5691-4337-bf93-4f4c79725a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:46.206533 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc51e6d-5691-4337-bf93-4f4c79725a5e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:46.206809 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc51e6d-5691-4337-bf93-4f4c79725a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:46.207571 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc51e6d-5691-4337-bf93-4f4c79725a5e None admin] PATH_INFO: `/v3/groups/4fd7c2055c3147f5ad7054d54f3a1ad2` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:46.208404 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc51e6d-5691-4337-bf93-4f4c79725a5e None admin] RBAC: Authorizing `identity:get_group(group_id=4fd7c2055c3147f5ad7054d54f3a1ad2)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:46.209946 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc51e6d-5691-4337-bf93-4f4c79725a5e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:46.211776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1225c0-5114-4359-af77-4aeafd8500a6 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:46.211925 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 174/361] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Jun 15 19:24:46 2022] GET /identity/v3/groups/4fd7c2055c3147f5ad7054d54f3a1ad2 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:46.213105 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf4f6b-e375-4f57-b446-fc82b8e5c975 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:46.213393 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf4f6b-e375-4f57-b446-fc82b8e5c975 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:46.213672 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf4f6b-e375-4f57-b446-fc82b8e5c975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:46.236862 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-48cf4f6b-e375-4f57-b446-fc82b8e5c975 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:46.238073 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-48cf4f6b-e375-4f57-b446-fc82b8e5c975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:46.257475 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edeabe2-998e-44c3-a40f-9e261e99de81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:46.260151 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edeabe2-998e-44c3-a40f-9e261e99de81 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:46.260151 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edeabe2-998e-44c3-a40f-9e261e99de81 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:46.260151 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edeabe2-998e-44c3-a40f-9e261e99de81 None admin] PATH_INFO: `/v3/users/952833e57cac4cf88e4d8e61c0f071e1` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:46.261163 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edeabe2-998e-44c3-a40f-9e261e99de81 None admin] RBAC: Authorizing `identity:get_user(user_id=952833e57cac4cf88e4d8e61c0f071e1)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:46.263318 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edeabe2-998e-44c3-a40f-9e261e99de81 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:46.265259 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 188/362] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:24:46 2022] GET /identity/v3/users/952833e57cac4cf88e4d8e61c0f071e1 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:46.266012 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edeabe2-998e-44c3-a40f-9e261e99de81 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:46.320562 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 175/363] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:46.322876 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cf4f6b-e375-4f57-b446-fc82b8e5c975 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:46.392187 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edeabe2-998e-44c3-a40f-9e261e99de81 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:46.396286 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:46.436841 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68b6882-29c5-41e5-9733-bea356917c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:46.437500 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b6882-29c5-41e5-9733-bea356917c17 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:46.437788 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b6882-29c5-41e5-9733-bea356917c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:46.438069 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b6882-29c5-41e5-9733-bea356917c17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:46.457845 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cf4f6b-e375-4f57-b446-fc82b8e5c975 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:46.470260 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:46.509360 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ba01d-7468-4ba3-bd9d-e18c942fdebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:46.510332 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ba01d-7468-4ba3-bd9d-e18c942fdebf None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:46.510622 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ba01d-7468-4ba3-bd9d-e18c942fdebf None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:46.510898 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ba01d-7468-4ba3-bd9d-e18c942fdebf None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:46.512311 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ba01d-7468-4ba3-bd9d-e18c942fdebf None admin] RBAC: Adding query filter params (name=member) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:46.512630 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ba01d-7468-4ba3-bd9d-e18c942fdebf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:46.514362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ba01d-7468-4ba3-bd9d-e18c942fdebf None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:46.524894 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 176/364] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jun 15 19:24:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:46.526077 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ba01d-7468-4ba3-bd9d-e18c942fdebf None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:46.533141 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68b6882-29c5-41e5-9733-bea356917c17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:46.534658 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68b6882-29c5-41e5-9733-bea356917c17 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:46.655393 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ba01d-7468-4ba3-bd9d-e18c942fdebf None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:46.661550 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 189/365] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:24:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 19:24:46.662282 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68b6882-29c5-41e5-9733-bea356917c17 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:46.674602 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:46.706702 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72913f3-b9ba-4117-9eac-35af537b2f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:46.707362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72913f3-b9ba-4117-9eac-35af537b2f63 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:46.707641 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72913f3-b9ba-4117-9eac-35af537b2f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:46.707914 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72913f3-b9ba-4117-9eac-35af537b2f63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:46.708762 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72913f3-b9ba-4117-9eac-35af537b2f63 None admin] RBAC: Adding query filter params (group.id=4fd7c2055c3147f5ad7054d54f3a1ad2, role.id=3920d5126e22480ab9a4eec9cf0fd4e7, scope.project.id=c4f989e5fad14bcebae5e279f22ee21d) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:46.709070 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72913f3-b9ba-4117-9eac-35af537b2f63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:46.710553 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72913f3-b9ba-4117-9eac-35af537b2f63 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:46.716794 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 177/366] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Jun 15 19:24:46 2022] GET /identity/v3/role_assignments?group.id=4fd7c2055c3147f5ad7054d54f3a1ad2&scope.project.id=c4f989e5fad14bcebae5e279f22ee21d&role.id=3920d5126e22480ab9a4eec9cf0fd4e7 => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:46.717602 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72913f3-b9ba-4117-9eac-35af537b2f63 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:46.742494 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:46.803451 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68b6882-29c5-41e5-9733-bea356917c17 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:46.831079 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72913f3-b9ba-4117-9eac-35af537b2f63 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:46.845930 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d1a29-deb2-46d0-9022-58ffd9b45cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:46.847004 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d1a29-deb2-46d0-9022-58ffd9b45cfa None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:46.847490 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d1a29-deb2-46d0-9022-58ffd9b45cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:46.847982 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d1a29-deb2-46d0-9022-58ffd9b45cfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:46.849214 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312d1a29-deb2-46d0-9022-58ffd9b45cfa None admin] RBAC: Adding query filter params (role.id=4e60cc1f881a4caebdc208088b7ffcaa, user.id=952833e57cac4cf88e4d8e61c0f071e1, scope.system=all) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:46.849741 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312d1a29-deb2-46d0-9022-58ffd9b45cfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:46.851467 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312d1a29-deb2-46d0-9022-58ffd9b45cfa None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:46.868778 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9e913-0f0d-4756-85b4-4ddf22ca3fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:46.870568 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9e913-0f0d-4756-85b4-4ddf22ca3fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:46.871976 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9e913-0f0d-4756-85b4-4ddf22ca3fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:46.872500 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 190/367] 10.222.0.15 () {56 vars in 1308 bytes} [Wed Jun 15 19:24:46 2022] GET /identity/v3/role_assignments?user.id=952833e57cac4cf88e4d8e61c0f071e1&scope.system=all&role.id=4e60cc1f881a4caebdc208088b7ffcaa => generated 510 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:46.872842 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9e913-0f0d-4756-85b4-4ddf22ca3fc8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:46.873343 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d1a29-deb2-46d0-9022-58ffd9b45cfa None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:46.886800 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad9e913-0f0d-4756-85b4-4ddf22ca3fc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:46.896366 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-6ad9e913-0f0d-4756-85b4-4ddf22ca3fc8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:24:46.896366 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:46.896366 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:46.896366 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:46.896366 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:46.896366 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:46.896366 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:46.896366 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:46.896366 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:46.896366 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:46.896366 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:46.896366 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:46.896366 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:46.896366 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:46.896366 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:46.896366 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:46.897309 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:46.897309 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:46.897309 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:46.897309 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:46.897309 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:46.897309 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:46.897309 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:46.897309 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:46.897309 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:46.897309 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:46.897309 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:46.897309 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:46.897309 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:46.897309 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:46.897309 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:46.897309 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:46.898151 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:46.898151 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:46.898151 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:46.898151 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:46.898151 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:46.898151 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:46.898151 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:46.898151 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:46.898151 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:46.898151 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:46.898151 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:24:46.898151 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:24:46.900980 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 178/368] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jun 15 19:24:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:46.902129 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9e913-0f0d-4756-85b4-4ddf22ca3fc8 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:46.976080 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:47.044882 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d1a29-deb2-46d0-9022-58ffd9b45cfa None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:47.061490 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:47.073109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9e913-0f0d-4756-85b4-4ddf22ca3fc8 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:47.088154 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61acd2d-3a63-4dfd-9cf6-eeed71e35645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:47.089043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61acd2d-3a63-4dfd-9cf6-eeed71e35645 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:47.089333 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61acd2d-3a63-4dfd-9cf6-eeed71e35645 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:47.090254 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61acd2d-3a63-4dfd-9cf6-eeed71e35645 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:47.128236 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfb4a92-558b-4f1a-b4c5-46d2203385bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:47.129814 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb4a92-558b-4f1a-b4c5-46d2203385bf None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:47.130442 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb4a92-558b-4f1a-b4c5-46d2203385bf None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:47.130928 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb4a92-558b-4f1a-b4c5-46d2203385bf None admin] PATH_INFO: `/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:47.132368 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfb4a92-558b-4f1a-b4c5-46d2203385bf None admin] RBAC: Authorizing `identity:get_project(project_id=383a1a7fd67a4b0c8baf1403a8b02f25)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:47.134632 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfb4a92-558b-4f1a-b4c5-46d2203385bf None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:47.137289 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 179/369] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:46 2022] GET /identity/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25 => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:47.138821 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfb4a92-558b-4f1a-b4c5-46d2203385bf None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:47.187818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61acd2d-3a63-4dfd-9cf6-eeed71e35645 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:47.189310 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61acd2d-3a63-4dfd-9cf6-eeed71e35645 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:47.293015 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfb4a92-558b-4f1a-b4c5-46d2203385bf None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:47.302891 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:47.312815 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 191/370] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:24:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 19:24:47.313962 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61acd2d-3a63-4dfd-9cf6-eeed71e35645 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:47.338023 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:47.356889 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348f2a4-fd63-42a9-8e1a-bf1317918996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:47.357580 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348f2a4-fd63-42a9-8e1a-bf1317918996 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:47.357866 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348f2a4-fd63-42a9-8e1a-bf1317918996 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:47.358169 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348f2a4-fd63-42a9-8e1a-bf1317918996 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:47.358945 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348f2a4-fd63-42a9-8e1a-bf1317918996 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:47.359407 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348f2a4-fd63-42a9-8e1a-bf1317918996 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:47.361124 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348f2a4-fd63-42a9-8e1a-bf1317918996 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:47.373126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 180/371] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:24:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:47.373971 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348f2a4-fd63-42a9-8e1a-bf1317918996 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:47.450214 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61acd2d-3a63-4dfd-9cf6-eeed71e35645 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:47.489047 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d43b9be-3dab-44b7-8ead-39c3ff118b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:47.490027 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43b9be-3dab-44b7-8ead-39c3ff118b09 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:47.490345 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43b9be-3dab-44b7-8ead-39c3ff118b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:47.490623 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43b9be-3dab-44b7-8ead-39c3ff118b09 None admin] PATH_INFO: `/v3/groups/4fd7c2055c3147f5ad7054d54f3a1ad2` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:47.491170 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348f2a4-fd63-42a9-8e1a-bf1317918996 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:47.491826 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d43b9be-3dab-44b7-8ead-39c3ff118b09 None admin] RBAC: Authorizing `identity:get_group(group_id=4fd7c2055c3147f5ad7054d54f3a1ad2)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:47.493407 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d43b9be-3dab-44b7-8ead-39c3ff118b09 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:47.495208 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 192/372] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Jun 15 19:24:47 2022] GET /identity/v3/groups/4fd7c2055c3147f5ad7054d54f3a1ad2 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:47.496065 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d43b9be-3dab-44b7-8ead-39c3ff118b09 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:47.535884 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2f8af-fd6a-4a16-bfb5-0350567be6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:47.537013 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2f8af-fd6a-4a16-bfb5-0350567be6fd None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:47.537414 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2f8af-fd6a-4a16-bfb5-0350567be6fd None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:47.537783 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2f8af-fd6a-4a16-bfb5-0350567be6fd None admin] PATH_INFO: `/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:47.539234 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e2f8af-fd6a-4a16-bfb5-0350567be6fd None admin] RBAC: Authorizing `identity:get_project(project_id=383a1a7fd67a4b0c8baf1403a8b02f25)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:47.540905 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e2f8af-fd6a-4a16-bfb5-0350567be6fd None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:47.546910 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 181/373] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:47 2022] GET /identity/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:47.546910 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2f8af-fd6a-4a16-bfb5-0350567be6fd None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:47.590025 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:47.606742 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:47.628960 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d43b9be-3dab-44b7-8ead-39c3ff118b09 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:47.666746 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773a927a-3a4a-4e8a-8ffc-a2f989bf4860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:47.668320 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a927a-3a4a-4e8a-8ffc-a2f989bf4860 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:47.668854 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a927a-3a4a-4e8a-8ffc-a2f989bf4860 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:47.669331 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a927a-3a4a-4e8a-8ffc-a2f989bf4860 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:47.671030 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2f8af-fd6a-4a16-bfb5-0350567be6fd None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:47.707262 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b9261-7cf9-4a4f-bbff-ecccfb53c633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:47.707806 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b9261-7cf9-4a4f-bbff-ecccfb53c633 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:47.708099 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b9261-7cf9-4a4f-bbff-ecccfb53c633 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:47.708381 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b9261-7cf9-4a4f-bbff-ecccfb53c633 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:47.709227 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b9261-7cf9-4a4f-bbff-ecccfb53c633 None admin] RBAC: Adding query filter params (group.id=4fd7c2055c3147f5ad7054d54f3a1ad2, role.id=3920d5126e22480ab9a4eec9cf0fd4e7, scope.project.id=383a1a7fd67a4b0c8baf1403a8b02f25) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:47.709548 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b9261-7cf9-4a4f-bbff-ecccfb53c633 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:47.711040 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b9261-7cf9-4a4f-bbff-ecccfb53c633 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:47.717444 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 182/374] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Jun 15 19:24:47 2022] GET /identity/v3/role_assignments?group.id=4fd7c2055c3147f5ad7054d54f3a1ad2&scope.project.id=383a1a7fd67a4b0c8baf1403a8b02f25&role.id=3920d5126e22480ab9a4eec9cf0fd4e7 => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:47.718316 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b9261-7cf9-4a4f-bbff-ecccfb53c633 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:47.746546 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773a927a-3a4a-4e8a-8ffc-a2f989bf4860 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:47.748765 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773a927a-3a4a-4e8a-8ffc-a2f989bf4860 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:47.828267 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:47.831849 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b9261-7cf9-4a4f-bbff-ecccfb53c633 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:47.840511 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:47.864078 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 193/375] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:24:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 19:24:47.870280 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773a927a-3a4a-4e8a-8ffc-a2f989bf4860 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:47.871281 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7764d6a7-fa0f-43a7-a2df-0adff2da1817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:47.872165 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764d6a7-fa0f-43a7-a2df-0adff2da1817 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:47.872453 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764d6a7-fa0f-43a7-a2df-0adff2da1817 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:47.872731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764d6a7-fa0f-43a7-a2df-0adff2da1817 None admin] PATH_INFO: `/v3/groups/4fd7c2055c3147f5ad7054d54f3a1ad2` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:47.874220 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7764d6a7-fa0f-43a7-a2df-0adff2da1817 None admin] RBAC: Authorizing `identity:get_group(group_id=4fd7c2055c3147f5ad7054d54f3a1ad2)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:47.876226 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7764d6a7-fa0f-43a7-a2df-0adff2da1817 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:47.877561 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 183/376] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Jun 15 19:24:47 2022] GET /identity/v3/groups/4fd7c2055c3147f5ad7054d54f3a1ad2 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:47.884055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7764d6a7-fa0f-43a7-a2df-0adff2da1817 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:47.989467 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773a927a-3a4a-4e8a-8ffc-a2f989bf4860 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:48.004340 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7764d6a7-fa0f-43a7-a2df-0adff2da1817 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:48.029541 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b795466-af40-43d4-9a81-d6ebadb6c4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:48.030912 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b795466-af40-43d4-9a81-d6ebadb6c4e8 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:48.031505 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b795466-af40-43d4-9a81-d6ebadb6c4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:48.031983 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b795466-af40-43d4-9a81-d6ebadb6c4e8 None admin] PATH_INFO: `/v3/projects/7d07be90a27c40718a346a1d9e5469b4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:48.033483 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b795466-af40-43d4-9a81-d6ebadb6c4e8 None admin] RBAC: Authorizing `identity:get_project(project_id=7d07be90a27c40718a346a1d9e5469b4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:48.035411 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b795466-af40-43d4-9a81-d6ebadb6c4e8 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:48.037916 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 194/377] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:47 2022] GET /identity/v3/projects/7d07be90a27c40718a346a1d9e5469b4 => generated 344 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:24:48.039469 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90961990-8ad4-4042-b51a-dbc47001d121 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:48.039976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90961990-8ad4-4042-b51a-dbc47001d121 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:48.040449 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90961990-8ad4-4042-b51a-dbc47001d121 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:48.046115 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450c946e-f767-47c0-952d-44f80d308a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:48.046115 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c946e-f767-47c0-952d-44f80d308a83 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:48.046115 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c946e-f767-47c0-952d-44f80d308a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:48.046115 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c946e-f767-47c0-952d-44f80d308a83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:48.046519 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450c946e-f767-47c0-952d-44f80d308a83 None admin] RBAC: Adding query filter params (group.id=4fd7c2055c3147f5ad7054d54f3a1ad2, role.id=e53a78912e764f8387e0cf637bb86bee, scope.project.id=383a1a7fd67a4b0c8baf1403a8b02f25) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:48.046519 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450c946e-f767-47c0-952d-44f80d308a83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:48.047030 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450c946e-f767-47c0-952d-44f80d308a83 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:48.047238 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 195/378] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:48.048986 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95012197-7c18-4914-b7ed-645d8d30bb7e None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:48.049485 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95012197-7c18-4914-b7ed-645d8d30bb7e None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:48.049966 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95012197-7c18-4914-b7ed-645d8d30bb7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:48.053819 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 184/379] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Jun 15 19:24:47 2022] GET /identity/v3/role_assignments?group.id=4fd7c2055c3147f5ad7054d54f3a1ad2&scope.project.id=383a1a7fd67a4b0c8baf1403a8b02f25&role.id=e53a78912e764f8387e0cf637bb86bee => generated 260 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:48.059298 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450c946e-f767-47c0-952d-44f80d308a83 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:48.069020 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-95012197-7c18-4914-b7ed-645d8d30bb7e None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:48.070701 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-95012197-7c18-4914-b7ed-645d8d30bb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:48.103850 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:48.130108 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:48.156136 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 196/380] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:48.180080 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95012197-7c18-4914-b7ed-645d8d30bb7e None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:48.184821 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450c946e-f767-47c0-952d-44f80d308a83 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:48.222645 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8667c16-f60b-4002-87e3-ec42cb0311b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:48.223853 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8667c16-f60b-4002-87e3-ec42cb0311b2 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:48.224300 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8667c16-f60b-4002-87e3-ec42cb0311b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:48.224841 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8667c16-f60b-4002-87e3-ec42cb0311b2 None admin] PATH_INFO: `/v3/groups/2f50a7a67b8e4e54b7289d1f6ba6a1c6` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:48.231048 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8667c16-f60b-4002-87e3-ec42cb0311b2 None admin] RBAC: Authorizing `identity:get_group(group_id=2f50a7a67b8e4e54b7289d1f6ba6a1c6)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:48.233027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8667c16-f60b-4002-87e3-ec42cb0311b2 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:48.235430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 185/381] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Jun 15 19:24:48 2022] GET /identity/v3/groups/2f50a7a67b8e4e54b7289d1f6ba6a1c6 => generated 229 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:48.241215 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8667c16-f60b-4002-87e3-ec42cb0311b2 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:48.291513 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95012197-7c18-4914-b7ed-645d8d30bb7e None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:48.331347 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a8dd9-90bd-4439-83ac-2ed9209fe6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:48.332363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a8dd9-90bd-4439-83ac-2ed9209fe6a1 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:48.332764 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a8dd9-90bd-4439-83ac-2ed9209fe6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:48.333132 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a8dd9-90bd-4439-83ac-2ed9209fe6a1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:48.345609 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a8dd9-90bd-4439-83ac-2ed9209fe6a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:48.353963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-ce0a8dd9-90bd-4439-83ac-2ed9209fe6a1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 19:24:48.353963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:48.353963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:48.353963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:48.353963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:48.353963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:48.353963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:48.353963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:48.353963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:48.353963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:48.353963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:48.353963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:48.353963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:48.353963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:48.353963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:48.353963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:48.354959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:48.354959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:48.354959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:48.354959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:48.354959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:48.354959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:48.354959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:48.354959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:48.354959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:48.354959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:48.354959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:48.354959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:48.354959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:48.354959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:48.354959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:48.354959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:48.356365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:48.356365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:48.356365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:48.356365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:48.356365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:48.356365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:48.356365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:48.356365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:48.356365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:48.356365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:48.356365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 19:24:48.356365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:24:48.357006 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:48.358158 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 197/382] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:24:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:24:48.364274 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a8dd9-90bd-4439-83ac-2ed9209fe6a1 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:48.365566 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8667c16-f60b-4002-87e3-ec42cb0311b2 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:48.410348 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13850c8d-eb4b-4ce8-84c6-b422e3121976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:48.411020 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13850c8d-eb4b-4ce8-84c6-b422e3121976 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:48.411309 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13850c8d-eb4b-4ce8-84c6-b422e3121976 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:48.411593 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13850c8d-eb4b-4ce8-84c6-b422e3121976 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:48.412442 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13850c8d-eb4b-4ce8-84c6-b422e3121976 None admin] RBAC: Adding query filter params (group.id=2f50a7a67b8e4e54b7289d1f6ba6a1c6, role.id=4f736313c26d4f8db48bbb2f346e50e4, scope.project.id=7d07be90a27c40718a346a1d9e5469b4) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:48.412760 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13850c8d-eb4b-4ce8-84c6-b422e3121976 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:48.414236 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13850c8d-eb4b-4ce8-84c6-b422e3121976 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:48.416552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:48.422888 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 186/383] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Jun 15 19:24:48 2022] GET /identity/v3/role_assignments?group.id=2f50a7a67b8e4e54b7289d1f6ba6a1c6&scope.project.id=7d07be90a27c40718a346a1d9e5469b4&role.id=4f736313c26d4f8db48bbb2f346e50e4 => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:48.484914 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a8dd9-90bd-4439-83ac-2ed9209fe6a1 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:48.522433 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3edc3ca-20ad-49a3-8e17-284feda8872a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:48.523106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3edc3ca-20ad-49a3-8e17-284feda8872a None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:48.523388 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3edc3ca-20ad-49a3-8e17-284feda8872a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:48.523661 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3edc3ca-20ad-49a3-8e17-284feda8872a None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:48.524416 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3edc3ca-20ad-49a3-8e17-284feda8872a None admin] RBAC: Adding query filter params (name=member) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:48.524723 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3edc3ca-20ad-49a3-8e17-284feda8872a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:48.526188 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3edc3ca-20ad-49a3-8e17-284feda8872a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:48.540815 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 198/384] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jun 15 19:24:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:48.541705 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13850c8d-eb4b-4ce8-84c6-b422e3121976 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:48.675554 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13850c8d-eb4b-4ce8-84c6-b422e3121976 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:48.709957 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ee55a-805c-4003-9183-73adf0779dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:48.710873 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ee55a-805c-4003-9183-73adf0779dfb None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:48.711161 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ee55a-805c-4003-9183-73adf0779dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:48.711444 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ee55a-805c-4003-9183-73adf0779dfb None admin] PATH_INFO: `/v3/groups/4fd7c2055c3147f5ad7054d54f3a1ad2` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:48.712663 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ee55a-805c-4003-9183-73adf0779dfb None admin] RBAC: Authorizing `identity:get_group(group_id=4fd7c2055c3147f5ad7054d54f3a1ad2)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:48.714153 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ee55a-805c-4003-9183-73adf0779dfb None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:48.716651 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 187/385] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Jun 15 19:24:48 2022] GET /identity/v3/groups/4fd7c2055c3147f5ad7054d54f3a1ad2 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:48.721940 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3edc3ca-20ad-49a3-8e17-284feda8872a None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:48.856429 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3edc3ca-20ad-49a3-8e17-284feda8872a None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:48.858068 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:48.905609 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a2cbc-eb39-4483-911a-a95dab99d007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:48.906519 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a2cbc-eb39-4483-911a-a95dab99d007 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:48.906801 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a2cbc-eb39-4483-911a-a95dab99d007 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:48.907073 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a2cbc-eb39-4483-911a-a95dab99d007 None admin] PATH_INFO: `/v3/projects/c4f989e5fad14bcebae5e279f22ee21d` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:48.908268 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a2cbc-eb39-4483-911a-a95dab99d007 None admin] RBAC: Authorizing `identity:get_project(project_id=c4f989e5fad14bcebae5e279f22ee21d)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:48.909705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a2cbc-eb39-4483-911a-a95dab99d007 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:48.911718 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 199/386] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:48 2022] GET /identity/v3/projects/c4f989e5fad14bcebae5e279f22ee21d => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:48.917445 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ee55a-805c-4003-9183-73adf0779dfb None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:48.953015 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:49.032261 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ee55a-805c-4003-9183-73adf0779dfb None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:49.067542 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39a16d0-f3cc-4af5-813a-9cbd65237e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:49.068798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a16d0-f3cc-4af5-813a-9cbd65237e86 None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:49.069326 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a16d0-f3cc-4af5-813a-9cbd65237e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:49.069810 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a16d0-f3cc-4af5-813a-9cbd65237e86 None admin] PATH_INFO: `/v3/projects/c4f989e5fad14bcebae5e279f22ee21d/groups/4fd7c2055c3147f5ad7054d54f3a1ad2/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:49.072029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39a16d0-f3cc-4af5-813a-9cbd65237e86 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4f989e5fad14bcebae5e279f22ee21d, group_id=4fd7c2055c3147f5ad7054d54f3a1ad2, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:49.073709 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39a16d0-f3cc-4af5-813a-9cbd65237e86 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:49.088333 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 188/387] 10.222.0.15 () {58 vars in 1442 bytes} [Wed Jun 15 19:24:48 2022] PUT /identity/v3/projects/c4f989e5fad14bcebae5e279f22ee21d/groups/4fd7c2055c3147f5ad7054d54f3a1ad2/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:24:49.653431 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc6637-4b09-4eed-af0d-7d52e06084aa None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:49.654441 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc6637-4b09-4eed-af0d-7d52e06084aa None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:49.655209 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc6637-4b09-4eed-af0d-7d52e06084aa None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:49.658386 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 200/388] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:49.664239 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e211a-fd28-442b-8645-b922ff08183c None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:49.664741 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e211a-fd28-442b-8645-b922ff08183c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:49.665231 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e211a-fd28-442b-8645-b922ff08183c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:49.707677 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-3c2e211a-fd28-442b-8645-b922ff08183c None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:49.709623 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3c2e211a-fd28-442b-8645-b922ff08183c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:49.857162 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 189/389] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:49.892047 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc6637-4b09-4eed-af0d-7d52e06084aa None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:50.084958 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc6637-4b09-4eed-af0d-7d52e06084aa None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:50.144116 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b7a695-b921-401b-80c7-8bc274f76d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:50.145458 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7a695-b921-401b-80c7-8bc274f76d4f None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:50.145952 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7a695-b921-401b-80c7-8bc274f76d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:50.146481 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7a695-b921-401b-80c7-8bc274f76d4f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:50.164561 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b7a695-b921-401b-80c7-8bc274f76d4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:50.181286 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-f8b7a695-b921-401b-80c7-8bc274f76d4f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 19:24:50.181286 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:50.181286 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:50.181286 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:50.181286 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:50.181286 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:50.181286 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:50.181286 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:50.181286 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:50.181286 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:50.181286 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:50.181286 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:50.181286 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:50.181286 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:50.181286 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:50.181286 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:50.183021 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:50.183021 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:50.183021 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:50.183021 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:50.183021 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:50.183021 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:50.183021 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:50.183021 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:50.183021 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:50.183021 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:50.183021 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:50.183021 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:50.183021 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:50.183021 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:50.183021 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:50.183021 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:50.184252 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:50.184252 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:50.184252 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:50.184252 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:50.184252 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:50.184252 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:50.184252 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:50.184252 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:50.184252 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:50.184252 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:50.184252 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 19:24:50.184252 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:24:50.184252 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 201/390] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jun 15 19:24:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:24:50.190138 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e211a-fd28-442b-8645-b922ff08183c None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:50.243159 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:50.375017 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e211a-fd28-442b-8645-b922ff08183c None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:50.435687 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8ecf23-507f-4e90-b9eb-a1ff5ec622c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:50.437290 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8ecf23-507f-4e90-b9eb-a1ff5ec622c1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:50.438050 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8ecf23-507f-4e90-b9eb-a1ff5ec622c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:50.438845 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8ecf23-507f-4e90-b9eb-a1ff5ec622c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:50.440549 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8ecf23-507f-4e90-b9eb-a1ff5ec622c1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:50.441370 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8ecf23-507f-4e90-b9eb-a1ff5ec622c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:50.444312 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8ecf23-507f-4e90-b9eb-a1ff5ec622c1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:50.463312 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 190/391] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:24:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:50.475854 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b7a695-b921-401b-80c7-8bc274f76d4f None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:50.532083 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:50.656987 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b7a695-b921-401b-80c7-8bc274f76d4f None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:50.716000 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce84ec-7ce9-4182-8b60-40cf65191dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:50.717503 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce84ec-7ce9-4182-8b60-40cf65191dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:50.718001 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce84ec-7ce9-4182-8b60-40cf65191dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:50.718517 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce84ec-7ce9-4182-8b60-40cf65191dc8 None admin] PATH_INFO: `/v3/groups/4fd7c2055c3147f5ad7054d54f3a1ad2` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:50.720508 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce84ec-7ce9-4182-8b60-40cf65191dc8 None admin] RBAC: Authorizing `identity:get_group(group_id=4fd7c2055c3147f5ad7054d54f3a1ad2)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:50.723225 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce84ec-7ce9-4182-8b60-40cf65191dc8 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:50.726660 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 202/392] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Jun 15 19:24:50 2022] GET /identity/v3/groups/4fd7c2055c3147f5ad7054d54f3a1ad2 => generated 226 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:50.735415 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8ecf23-507f-4e90-b9eb-a1ff5ec622c1 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:50.876726 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:50.895935 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e469371-a850-4477-8fe6-534d7b81e3b8 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:50.896429 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e469371-a850-4477-8fe6-534d7b81e3b8 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:50.896528 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e469371-a850-4477-8fe6-534d7b81e3b8 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:50.898023 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 203/393] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 19:24:50.902978 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b4973-5b35-42ee-8cd5-cd045b852902 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:50.903536 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b4973-5b35-42ee-8cd5-cd045b852902 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:50.904035 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b4973-5b35-42ee-8cd5-cd045b852902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:50.905325 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8ecf23-507f-4e90-b9eb-a1ff5ec622c1 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:50.939396 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-102b4973-5b35-42ee-8cd5-cd045b852902 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:50.941018 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-102b4973-5b35-42ee-8cd5-cd045b852902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:50.945020 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc63c9a0-4c82-4486-b61c-9b74592c4bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:50.945903 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63c9a0-4c82-4486-b61c-9b74592c4bef None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:50.946261 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63c9a0-4c82-4486-b61c-9b74592c4bef None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:50.946548 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63c9a0-4c82-4486-b61c-9b74592c4bef None admin] PATH_INFO: `/v3/projects/c4f989e5fad14bcebae5e279f22ee21d` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:50.947763 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc63c9a0-4c82-4486-b61c-9b74592c4bef None admin] RBAC: Authorizing `identity:get_project(project_id=c4f989e5fad14bcebae5e279f22ee21d)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:50.949289 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc63c9a0-4c82-4486-b61c-9b74592c4bef None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:50.954108 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 191/394] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:50 2022] GET /identity/v3/projects/c4f989e5fad14bcebae5e279f22ee21d => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:50.956996 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc63c9a0-4c82-4486-b61c-9b74592c4bef None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:51.085551 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 204/395] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:51.107961 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc63c9a0-4c82-4486-b61c-9b74592c4bef None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:51.119484 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102b4973-5b35-42ee-8cd5-cd045b852902 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:51.161286 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:51.178835 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989970f-0d34-49a9-b5da-167571f15c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:51.179739 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989970f-0d34-49a9-b5da-167571f15c96 None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:51.180023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989970f-0d34-49a9-b5da-167571f15c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:51.180297 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989970f-0d34-49a9-b5da-167571f15c96 None admin] PATH_INFO: `/v3/projects/c4f989e5fad14bcebae5e279f22ee21d/groups/4fd7c2055c3147f5ad7054d54f3a1ad2/roles/3920d5126e22480ab9a4eec9cf0fd4e7` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:51.190608 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2989970f-0d34-49a9-b5da-167571f15c96 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4f989e5fad14bcebae5e279f22ee21d, group_id=4fd7c2055c3147f5ad7054d54f3a1ad2, role_id=3920d5126e22480ab9a4eec9cf0fd4e7)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:51.192007 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2989970f-0d34-49a9-b5da-167571f15c96 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:51.213937 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 192/396] 10.222.0.15 () {58 vars in 1442 bytes} [Wed Jun 15 19:24:50 2022] PUT /identity/v3/projects/c4f989e5fad14bcebae5e279f22ee21d/groups/4fd7c2055c3147f5ad7054d54f3a1ad2/roles/3920d5126e22480ab9a4eec9cf0fd4e7 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:24:51.259095 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6528070-2d8a-437d-8195-aed437c14f83 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:51.259456 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6528070-2d8a-437d-8195-aed437c14f83 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:51.259689 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6528070-2d8a-437d-8195-aed437c14f83 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:51.261989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 193/397] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:51.267672 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62fb24d-24f7-4900-8eee-dccdba7b32e5 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:51.267963 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62fb24d-24f7-4900-8eee-dccdba7b32e5 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:51.268246 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62fb24d-24f7-4900-8eee-dccdba7b32e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:51.268430 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102b4973-5b35-42ee-8cd5-cd045b852902 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:51.286678 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c62fb24d-24f7-4900-8eee-dccdba7b32e5 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:51.287874 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c62fb24d-24f7-4900-8eee-dccdba7b32e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:51.298556 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:51.307060 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a5912-f2c1-4059-b75d-39cffbb65b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:51.308033 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a5912-f2c1-4059-b75d-39cffbb65b5d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:51.308281 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a5912-f2c1-4059-b75d-39cffbb65b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:51.308560 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a5912-f2c1-4059-b75d-39cffbb65b5d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:51.335367 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511a5912-f2c1-4059-b75d-39cffbb65b5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:51.353056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-511a5912-f2c1-4059-b75d-39cffbb65b5d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 19:24:51.353056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:51.353056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:51.353056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:51.353056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:51.353056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:51.353056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:51.353056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:51.353056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:51.353056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:51.353056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:51.353056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:51.353056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:51.353056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:51.353056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:51.353056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:51.354072 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:51.354072 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:51.354072 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:51.354072 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:51.354072 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:51.354072 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:51.354072 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:51.354072 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:51.354072 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:51.354072 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:51.354072 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:51.354072 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:51.354072 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:51.354072 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:51.354072 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:51.354072 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:51.354898 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:51.354898 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:51.354898 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:51.354898 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:51.354898 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:51.354898 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:51.354898 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:51.354898 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:51.354898 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:51.354898 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:51.354898 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 19:24:51.354898 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:24:51.358129 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 205/398] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jun 15 19:24:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:51.362673 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a5912-f2c1-4059-b75d-39cffbb65b5d None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:51.457725 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 194/399] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 19:24:51.459567 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced20168-9ebb-4770-a002-1de1971b799a None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:51.461100 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced20168-9ebb-4770-a002-1de1971b799a None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:51.461589 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced20168-9ebb-4770-a002-1de1971b799a None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:51.463151 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 195/400] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:51.472016 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911021b-512c-422c-9632-a8376724eaba None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:51.472550 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911021b-512c-422c-9632-a8376724eaba None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:51.473042 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911021b-512c-422c-9632-a8376724eaba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:51.491282 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-1911021b-512c-422c-9632-a8376724eaba None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:51.492570 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1911021b-512c-422c-9632-a8376724eaba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:51.500971 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:51.515333 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a5912-f2c1-4059-b75d-39cffbb65b5d None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:51.566149 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 196/401] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:51.568966 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187df1c4-448f-49a1-a63f-01b40d1359d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:51.569819 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187df1c4-448f-49a1-a63f-01b40d1359d6 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:51.570295 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187df1c4-448f-49a1-a63f-01b40d1359d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:51.570724 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187df1c4-448f-49a1-a63f-01b40d1359d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:51.571660 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187df1c4-448f-49a1-a63f-01b40d1359d6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:51.572123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187df1c4-448f-49a1-a63f-01b40d1359d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:51.573645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187df1c4-448f-49a1-a63f-01b40d1359d6 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:51.575312 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911021b-512c-422c-9632-a8376724eaba None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:51.585491 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 206/402] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:24:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:51.593068 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187df1c4-448f-49a1-a63f-01b40d1359d6 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:51.673207 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:51.716291 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187df1c4-448f-49a1-a63f-01b40d1359d6 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:51.720327 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911021b-512c-422c-9632-a8376724eaba None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:51.759529 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e39326e-3e53-4fcf-95af-f4c53a66ade0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:51.760470 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39326e-3e53-4fcf-95af-f4c53a66ade0 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:51.760901 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39326e-3e53-4fcf-95af-f4c53a66ade0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:51.761366 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39326e-3e53-4fcf-95af-f4c53a66ade0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:51.762910 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9f5303-2f69-47e9-8dac-68ced775900a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:51.764608 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f5303-2f69-47e9-8dac-68ced775900a None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:51.765091 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f5303-2f69-47e9-8dac-68ced775900a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:51.765544 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f5303-2f69-47e9-8dac-68ced775900a None admin] PATH_INFO: `/v3/groups/4fd7c2055c3147f5ad7054d54f3a1ad2` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:51.767236 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9f5303-2f69-47e9-8dac-68ced775900a None admin] RBAC: Authorizing `identity:get_group(group_id=4fd7c2055c3147f5ad7054d54f3a1ad2)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:51.768887 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9f5303-2f69-47e9-8dac-68ced775900a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:51.771589 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 207/403] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Jun 15 19:24:51 2022] GET /identity/v3/groups/4fd7c2055c3147f5ad7054d54f3a1ad2 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:51.772534 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9f5303-2f69-47e9-8dac-68ced775900a None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:51.778386 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e39326e-3e53-4fcf-95af-f4c53a66ade0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:51.786636 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-4e39326e-3e53-4fcf-95af-f4c53a66ade0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 19:24:51.786636 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:51.786636 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:51.786636 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:51.786636 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:51.786636 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:51.786636 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:51.786636 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:51.786636 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:51.786636 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:51.786636 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:51.786636 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:51.786636 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:51.786636 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:51.786636 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:51.786636 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:51.787549 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:51.787549 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:51.787549 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:51.787549 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:51.787549 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:51.787549 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:51.787549 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:51.787549 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:51.787549 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:51.787549 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:51.787549 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:51.787549 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:51.787549 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:51.787549 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:51.787549 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:51.787549 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:51.788326 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:51.788326 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:51.788326 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:51.788326 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:51.788326 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:51.788326 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:51.788326 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:51.788326 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:51.788326 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:51.788326 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:51.788326 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 19:24:51.788326 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:24:51.793702 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 197/404] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:24:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:51.795070 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e39326e-3e53-4fcf-95af-f4c53a66ade0 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:51.797479 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:51.903717 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9f5303-2f69-47e9-8dac-68ced775900a None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:51.915080 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e39326e-3e53-4fcf-95af-f4c53a66ade0 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:51.940792 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea23272-9481-4a3f-82c2-5e8f4b07c01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:51.941726 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea23272-9481-4a3f-82c2-5e8f4b07c01d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:51.942142 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea23272-9481-4a3f-82c2-5e8f4b07c01d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:51.942465 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea23272-9481-4a3f-82c2-5e8f4b07c01d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:51.944772 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:51.954833 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea23272-9481-4a3f-82c2-5e8f4b07c01d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:51.955132 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951d54f-ba39-47e7-bdf5-b7e0c4559eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:51.956195 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951d54f-ba39-47e7-bdf5-b7e0c4559eca None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:51.956652 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951d54f-ba39-47e7-bdf5-b7e0c4559eca None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:51.957176 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951d54f-ba39-47e7-bdf5-b7e0c4559eca None admin] PATH_INFO: `/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:51.958596 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3951d54f-ba39-47e7-bdf5-b7e0c4559eca None admin] RBAC: Authorizing `identity:get_project(project_id=383a1a7fd67a4b0c8baf1403a8b02f25)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:51.962350 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3951d54f-ba39-47e7-bdf5-b7e0c4559eca None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:51.964050 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-2ea23272-9481-4a3f-82c2-5e8f4b07c01d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:24:51.964050 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:51.964050 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:51.964050 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:51.964050 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:51.964050 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:51.964050 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:51.964050 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:51.964050 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:51.964050 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:51.964050 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:51.964050 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:51.964050 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:51.964050 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:51.964050 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:51.964050 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:51.965486 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:51.965486 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:51.965486 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:51.965486 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:51.965486 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:51.965486 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:51.965486 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:51.965486 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:51.965486 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:51.965486 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:51.965486 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:51.965486 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:51.965486 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:51.965486 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:51.965486 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:51.965486 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:51.966356 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:51.966356 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:51.966356 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:51.966356 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:51.966356 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:51.966356 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:51.966356 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:51.966356 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:51.966356 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:51.966356 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:51.966356 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:24:51.966356 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:24:51.967999 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 198/405] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:51 2022] GET /identity/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:51.969008 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951d54f-ba39-47e7-bdf5-b7e0c4559eca None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:51.973464 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 208/406] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jun 15 19:24:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:51.973906 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea23272-9481-4a3f-82c2-5e8f4b07c01d None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:52.064206 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:52.088131 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea23272-9481-4a3f-82c2-5e8f4b07c01d None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:52.088656 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951d54f-ba39-47e7-bdf5-b7e0c4559eca None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:52.122836 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2a7c22-86be-403e-9cc5-54a62f51584f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:52.123309 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89395d9a-15d9-4f68-82b6-eaa177ea3be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:52.123723 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a7c22-86be-403e-9cc5-54a62f51584f None admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:52.123973 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89395d9a-15d9-4f68-82b6-eaa177ea3be7 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:52.124068 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a7c22-86be-403e-9cc5-54a62f51584f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:52.124256 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89395d9a-15d9-4f68-82b6-eaa177ea3be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:52.124360 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a7c22-86be-403e-9cc5-54a62f51584f None admin] PATH_INFO: `/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25/groups/4fd7c2055c3147f5ad7054d54f3a1ad2/roles/3920d5126e22480ab9a4eec9cf0fd4e7` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:52.124530 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89395d9a-15d9-4f68-82b6-eaa177ea3be7 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:52.125285 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89395d9a-15d9-4f68-82b6-eaa177ea3be7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:52.125585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89395d9a-15d9-4f68-82b6-eaa177ea3be7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:52.126637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2a7c22-86be-403e-9cc5-54a62f51584f None admin] RBAC: Authorizing `identity:create_grant(project_id=383a1a7fd67a4b0c8baf1403a8b02f25, group_id=4fd7c2055c3147f5ad7054d54f3a1ad2, role_id=3920d5126e22480ab9a4eec9cf0fd4e7)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:52.126989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89395d9a-15d9-4f68-82b6-eaa177ea3be7 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:52.128051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2a7c22-86be-403e-9cc5-54a62f51584f None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:52.137609 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 199/407] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jun 15 19:24:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:52.138873 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89395d9a-15d9-4f68-82b6-eaa177ea3be7 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:52.147781 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 209/408] 10.222.0.15 () {58 vars in 1442 bytes} [Wed Jun 15 19:24:51 2022] PUT /identity/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25/groups/4fd7c2055c3147f5ad7054d54f3a1ad2/roles/3920d5126e22480ab9a4eec9cf0fd4e7 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:24:52.149430 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44a6dc-02cc-4dfb-b1d3-4815cf9a0a01 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:52.149723 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44a6dc-02cc-4dfb-b1d3-4815cf9a0a01 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:52.149997 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44a6dc-02cc-4dfb-b1d3-4815cf9a0a01 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:52.151442 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 210/409] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:52.152173 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e44a6dc-02cc-4dfb-b1d3-4815cf9a0a01 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:52.226464 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:52.280163 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89395d9a-15d9-4f68-82b6-eaa177ea3be7 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:52.295086 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e44a6dc-02cc-4dfb-b1d3-4815cf9a0a01 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:52.330308 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf5bee1-c1f6-4edd-9692-e345cd383337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:52.330987 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf5bee1-c1f6-4edd-9692-e345cd383337 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:52.331265 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf5bee1-c1f6-4edd-9692-e345cd383337 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:52.331589 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf5bee1-c1f6-4edd-9692-e345cd383337 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:52.332296 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf5bee1-c1f6-4edd-9692-e345cd383337 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:52.332608 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf5bee1-c1f6-4edd-9692-e345cd383337 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:52.333977 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf5bee1-c1f6-4edd-9692-e345cd383337 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:52.347311 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 200/410] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:24:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:52.349139 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd62a01-b969-41e3-8a6c-01fe59173adb None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:52.349139 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd62a01-b969-41e3-8a6c-01fe59173adb None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:52.349302 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd62a01-b969-41e3-8a6c-01fe59173adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:52.352687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b63911-f22f-4b41-929e-b681d844ca1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:52.353793 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b63911-f22f-4b41-929e-b681d844ca1e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:52.353895 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b63911-f22f-4b41-929e-b681d844ca1e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:52.354110 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b63911-f22f-4b41-929e-b681d844ca1e None admin] PATH_INFO: `/v3/groups/4fd7c2055c3147f5ad7054d54f3a1ad2` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:52.356398 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b63911-f22f-4b41-929e-b681d844ca1e None admin] RBAC: Authorizing `identity:get_group(group_id=4fd7c2055c3147f5ad7054d54f3a1ad2)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:52.357871 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b63911-f22f-4b41-929e-b681d844ca1e None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:52.362108 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 211/411] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Jun 15 19:24:52 2022] GET /identity/v3/groups/4fd7c2055c3147f5ad7054d54f3a1ad2 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:52.362108 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b63911-f22f-4b41-929e-b681d844ca1e None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:52.375107 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:52.376989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-0fd62a01-b969-41e3-8a6c-01fe59173adb None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:52.378514 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0fd62a01-b969-41e3-8a6c-01fe59173adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:52.455699 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 201/412] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:52.456831 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd62a01-b969-41e3-8a6c-01fe59173adb None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:52.480260 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b63911-f22f-4b41-929e-b681d844ca1e None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:52.512459 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:52.529636 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2b26b-606c-4500-8255-23abb397c077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:52.530893 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2b26b-606c-4500-8255-23abb397c077 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:52.531250 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2b26b-606c-4500-8255-23abb397c077 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:52.531598 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2b26b-606c-4500-8255-23abb397c077 None admin] PATH_INFO: `/v3/groups/2f50a7a67b8e4e54b7289d1f6ba6a1c6` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:52.534120 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a2b26b-606c-4500-8255-23abb397c077 None admin] RBAC: Authorizing `identity:get_group(group_id=2f50a7a67b8e4e54b7289d1f6ba6a1c6)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:52.535706 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a2b26b-606c-4500-8255-23abb397c077 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:52.537740 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 212/413] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Jun 15 19:24:52 2022] GET /identity/v3/groups/2f50a7a67b8e4e54b7289d1f6ba6a1c6 => generated 229 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:52.538608 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2b26b-606c-4500-8255-23abb397c077 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:52.570207 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd62a01-b969-41e3-8a6c-01fe59173adb None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:52.606386 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d782a-f142-46b3-b978-61bbd00b3de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:52.607368 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d782a-f142-46b3-b978-61bbd00b3de1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:52.607721 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d782a-f142-46b3-b978-61bbd00b3de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:52.608066 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d782a-f142-46b3-b978-61bbd00b3de1 None admin] PATH_INFO: `/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:52.610262 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3d782a-f142-46b3-b978-61bbd00b3de1 None admin] RBAC: Authorizing `identity:get_project(project_id=383a1a7fd67a4b0c8baf1403a8b02f25)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:52.610931 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3d782a-f142-46b3-b978-61bbd00b3de1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:52.613807 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 202/414] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:52 2022] GET /identity/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:52.614925 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d782a-f142-46b3-b978-61bbd00b3de1 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:52.642539 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:52.671898 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2b26b-606c-4500-8255-23abb397c077 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:52.707510 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c4f3bb-bbdf-42c6-94a1-9a0d7de370f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:52.708334 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c4f3bb-bbdf-42c6-94a1-9a0d7de370f6 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:52.708617 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c4f3bb-bbdf-42c6-94a1-9a0d7de370f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:52.708895 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c4f3bb-bbdf-42c6-94a1-9a0d7de370f6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:52.723356 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c4f3bb-bbdf-42c6-94a1-9a0d7de370f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:52.731209 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-b2c4f3bb-bbdf-42c6-94a1-9a0d7de370f6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 19:24:52.731209 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:52.731209 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:52.731209 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:52.731209 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:52.731209 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:52.731209 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:52.731209 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:52.731209 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:52.731209 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:52.731209 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:52.731209 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:52.731209 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:52.731209 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:52.731209 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:52.731209 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:52.732174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:52.732174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:52.732174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:52.732174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:52.732174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:52.732174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:52.732174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:52.732174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:52.732174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:52.732174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:52.732174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:52.732174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:52.732174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:52.732174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:52.732174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:52.732174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:52.732984 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:52.732984 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:52.732984 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:52.732984 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:52.732984 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:52.732984 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:52.732984 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:52.732984 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:52.732984 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:52.732984 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:52.732984 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 19:24:52.732984 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:24:52.737336 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 213/415] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:24:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:52.738163 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c4f3bb-bbdf-42c6-94a1-9a0d7de370f6 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:52.755831 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d782a-f142-46b3-b978-61bbd00b3de1 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:52.780328 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:52.805331 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc50c8e-0e33-40d0-ae13-0fc79e459048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:52.806519 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc50c8e-0e33-40d0-ae13-0fc79e459048 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:52.806998 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc50c8e-0e33-40d0-ae13-0fc79e459048 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:52.807708 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc50c8e-0e33-40d0-ae13-0fc79e459048 None admin] PATH_INFO: `/v3/projects/7d07be90a27c40718a346a1d9e5469b4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:52.809732 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc50c8e-0e33-40d0-ae13-0fc79e459048 None admin] RBAC: Authorizing `identity:get_project(project_id=7d07be90a27c40718a346a1d9e5469b4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:52.811891 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc50c8e-0e33-40d0-ae13-0fc79e459048 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:52.820437 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 203/416] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:52 2022] GET /identity/v3/projects/7d07be90a27c40718a346a1d9e5469b4 => generated 344 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:52.820437 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc50c8e-0e33-40d0-ae13-0fc79e459048 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:52.848861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c4f3bb-bbdf-42c6-94a1-9a0d7de370f6 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:52.897127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e62c2-19aa-4f43-8346-49ec1d3ff3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:52.899055 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e62c2-19aa-4f43-8346-49ec1d3ff3c0 None admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:52.899629 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e62c2-19aa-4f43-8346-49ec1d3ff3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:52.900360 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e62c2-19aa-4f43-8346-49ec1d3ff3c0 None admin] PATH_INFO: `/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25/groups/4fd7c2055c3147f5ad7054d54f3a1ad2/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:52.903984 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e62c2-19aa-4f43-8346-49ec1d3ff3c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=383a1a7fd67a4b0c8baf1403a8b02f25, group_id=4fd7c2055c3147f5ad7054d54f3a1ad2, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:52.906782 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e62c2-19aa-4f43-8346-49ec1d3ff3c0 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:52.910108 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:52.927788 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 214/417] 10.222.0.15 () {58 vars in 1442 bytes} [Wed Jun 15 19:24:52 2022] PUT /identity/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25/groups/4fd7c2055c3147f5ad7054d54f3a1ad2/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:24:52.929077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e62c2-19aa-4f43-8346-49ec1d3ff3c0 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:52.950710 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc50c8e-0e33-40d0-ae13-0fc79e459048 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:52.992405 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac213b05-d926-4525-b6e3-3a370298f6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:52.993088 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac213b05-d926-4525-b6e3-3a370298f6d1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:52.993373 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac213b05-d926-4525-b6e3-3a370298f6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:52.993702 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac213b05-d926-4525-b6e3-3a370298f6d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:52.994534 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac213b05-d926-4525-b6e3-3a370298f6d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:52.994843 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac213b05-d926-4525-b6e3-3a370298f6d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:52.996958 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac213b05-d926-4525-b6e3-3a370298f6d1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:53.007139 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 204/418] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jun 15 19:24:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:53.013481 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac213b05-d926-4525-b6e3-3a370298f6d1 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:53.050071 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e62c2-19aa-4f43-8346-49ec1d3ff3c0 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:53.069418 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:53.088519 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae5eaee-938d-4dc7-bf38-177e4a5eb70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:53.089859 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5eaee-938d-4dc7-bf38-177e4a5eb70f None admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:53.090366 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5eaee-938d-4dc7-bf38-177e4a5eb70f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:53.090858 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5eaee-938d-4dc7-bf38-177e4a5eb70f None admin] PATH_INFO: `/v3/projects/7d07be90a27c40718a346a1d9e5469b4/groups/2f50a7a67b8e4e54b7289d1f6ba6a1c6/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:53.093274 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae5eaee-938d-4dc7-bf38-177e4a5eb70f None admin] RBAC: Authorizing `identity:create_grant(project_id=7d07be90a27c40718a346a1d9e5469b4, group_id=2f50a7a67b8e4e54b7289d1f6ba6a1c6, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:53.095124 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae5eaee-938d-4dc7-bf38-177e4a5eb70f None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:53.137303 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac213b05-d926-4525-b6e3-3a370298f6d1 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:53.138121 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 215/419] 10.222.0.15 () {58 vars in 1442 bytes} [Wed Jun 15 19:24:52 2022] PUT /identity/v3/projects/7d07be90a27c40718a346a1d9e5469b4/groups/2f50a7a67b8e4e54b7289d1f6ba6a1c6/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:24:53.202843 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db577a1e-48b3-4504-b3fb-4aabfd69d888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:53.203525 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db577a1e-48b3-4504-b3fb-4aabfd69d888 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:53.203806 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db577a1e-48b3-4504-b3fb-4aabfd69d888 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:53.204082 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db577a1e-48b3-4504-b3fb-4aabfd69d888 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:53.306231 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db577a1e-48b3-4504-b3fb-4aabfd69d888 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:53.307630 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db577a1e-48b3-4504-b3fb-4aabfd69d888 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:53.375907 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:53.450008 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 205/420] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:24:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 19:24:53.456611 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae5eaee-938d-4dc7-bf38-177e4a5eb70f None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:53.568626 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:53.588158 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c4d6c-0f1c-4657-b159-96ac7e39a161 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:53.588490 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c4d6c-0f1c-4657-b159-96ac7e39a161 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:53.588725 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c4d6c-0f1c-4657-b159-96ac7e39a161 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:53.590437 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 206/421] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:53.595513 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5c0ea-2a43-47b0-8521-756a817d799e None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:53.595812 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5c0ea-2a43-47b0-8521-756a817d799e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:53.596065 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5c0ea-2a43-47b0-8521-756a817d799e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:53.623753 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-d4d5c0ea-2a43-47b0-8521-756a817d799e None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:53.625300 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d4d5c0ea-2a43-47b0-8521-756a817d799e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:53.641361 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae5eaee-938d-4dc7-bf38-177e4a5eb70f None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:53.698032 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 207/422] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:53.712861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ed23f-b22b-49f3-a226-9b0b483bf307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:53.712861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ed23f-b22b-49f3-a226-9b0b483bf307 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:53.712861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ed23f-b22b-49f3-a226-9b0b483bf307 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:53.712861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ed23f-b22b-49f3-a226-9b0b483bf307 None admin] PATH_INFO: `/v3/projects/c4f989e5fad14bcebae5e279f22ee21d` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:53.713938 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ed23f-b22b-49f3-a226-9b0b483bf307 None admin] RBAC: Authorizing `identity:get_project(project_id=c4f989e5fad14bcebae5e279f22ee21d)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:53.716087 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ed23f-b22b-49f3-a226-9b0b483bf307 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:53.718752 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 216/423] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:53 2022] GET /identity/v3/projects/c4f989e5fad14bcebae5e279f22ee21d => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:53.725778 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5c0ea-2a43-47b0-8521-756a817d799e None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:53.733562 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ed23f-b22b-49f3-a226-9b0b483bf307 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:53.857259 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ed23f-b22b-49f3-a226-9b0b483bf307 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:53.885484 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:53.899678 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6cb51c-f666-4e20-aab6-3b6d302c417f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:53.901325 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6cb51c-f666-4e20-aab6-3b6d302c417f None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:53.902449 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6cb51c-f666-4e20-aab6-3b6d302c417f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:53.903518 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6cb51c-f666-4e20-aab6-3b6d302c417f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:53.917459 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:53.919098 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6cb51c-f666-4e20-aab6-3b6d302c417f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:53.930495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-9a6cb51c-f666-4e20-aab6-3b6d302c417f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 19:24:53.930495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:53.930495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:53.930495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:53.930495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:53.930495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:53.930495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:53.930495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:53.930495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:53.930495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:53.930495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:53.930495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:53.930495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:53.930495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:53.930495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:53.930495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:53.931944 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:53.931944 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:53.931944 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:53.931944 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:53.931944 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:53.931944 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:53.931944 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:53.931944 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:53.931944 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:53.931944 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:53.931944 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:53.931944 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:53.931944 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:53.931944 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:53.931944 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:53.931944 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:53.934224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:53.934224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:53.934224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:53.934224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:53.934224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:53.934224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:53.934224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:53.934224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:53.934224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:53.934224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:53.934224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 19:24:53.934224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:24:53.937219 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 217/424] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jun 15 19:24:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:24:53.943134 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6cb51c-f666-4e20-aab6-3b6d302c417f None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:53.979396 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5c0ea-2a43-47b0-8521-756a817d799e None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:54.050584 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c60e09-e418-476a-a643-b26852f78e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:54.051437 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c60e09-e418-476a-a643-b26852f78e22 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:54.051720 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c60e09-e418-476a-a643-b26852f78e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:54.051990 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c60e09-e418-476a-a643-b26852f78e22 None admin] PATH_INFO: `/v3/groups/4fd7c2055c3147f5ad7054d54f3a1ad2` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:54.053747 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c60e09-e418-476a-a643-b26852f78e22 None admin] RBAC: Authorizing `identity:get_group(group_id=4fd7c2055c3147f5ad7054d54f3a1ad2)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:54.055228 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c60e09-e418-476a-a643-b26852f78e22 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:54.058697 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 208/425] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Jun 15 19:24:53 2022] GET /identity/v3/groups/4fd7c2055c3147f5ad7054d54f3a1ad2 => generated 226 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:54.063564 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c60e09-e418-476a-a643-b26852f78e22 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:54.164067 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6cb51c-f666-4e20-aab6-3b6d302c417f None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:54.204053 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c60e09-e418-476a-a643-b26852f78e22 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:54.244825 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad7df98-ea30-4f12-b87b-f14bbdafe7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:54.245580 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7df98-ea30-4f12-b87b-f14bbdafe7ed None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:54.245869 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7df98-ea30-4f12-b87b-f14bbdafe7ed None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:54.246163 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7df98-ea30-4f12-b87b-f14bbdafe7ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:54.247003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad7df98-ea30-4f12-b87b-f14bbdafe7ed None admin] RBAC: Adding query filter params (group.id=4fd7c2055c3147f5ad7054d54f3a1ad2, role.id=e53a78912e764f8387e0cf637bb86bee, scope.project.id=c4f989e5fad14bcebae5e279f22ee21d) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:54.247312 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad7df98-ea30-4f12-b87b-f14bbdafe7ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:54.248740 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad7df98-ea30-4f12-b87b-f14bbdafe7ed None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:54.252498 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb5d13-87d2-437f-910a-6e85e5cbf55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:54.253218 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb5d13-87d2-437f-910a-6e85e5cbf55e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:54.253504 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb5d13-87d2-437f-910a-6e85e5cbf55e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:54.253776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb5d13-87d2-437f-910a-6e85e5cbf55e None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:54.254546 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cb5d13-87d2-437f-910a-6e85e5cbf55e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:54.254852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cb5d13-87d2-437f-910a-6e85e5cbf55e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:54.256196 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cb5d13-87d2-437f-910a-6e85e5cbf55e None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:54.263948 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 209/426] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Jun 15 19:24:54 2022] GET /identity/v3/role_assignments?group.id=4fd7c2055c3147f5ad7054d54f3a1ad2&scope.project.id=c4f989e5fad14bcebae5e279f22ee21d&role.id=e53a78912e764f8387e0cf637bb86bee => generated 611 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:54.271235 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 218/427] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:24:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:54.277569 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad7df98-ea30-4f12-b87b-f14bbdafe7ed None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:54.291592 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:54.417166 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad7df98-ea30-4f12-b87b-f14bbdafe7ed None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:54.455055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0155f0a-b114-4c78-bf64-cece3ad2a57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:54.455731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0155f0a-b114-4c78-bf64-cece3ad2a57d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:54.456016 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0155f0a-b114-4c78-bf64-cece3ad2a57d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:54.456294 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0155f0a-b114-4c78-bf64-cece3ad2a57d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:54.530498 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0155f0a-b114-4c78-bf64-cece3ad2a57d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:54.532346 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0155f0a-b114-4c78-bf64-cece3ad2a57d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:54.543831 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:54.554171 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0520ea8e-100d-49bc-9704-b6a70376e606 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:54.554467 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0520ea8e-100d-49bc-9704-b6a70376e606 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:54.554746 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0520ea8e-100d-49bc-9704-b6a70376e606 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:54.556109 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 219/428] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:54.562447 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d510e-31bc-4420-900e-f5efcdd851d5 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:54.562999 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d510e-31bc-4420-900e-f5efcdd851d5 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:54.563474 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d510e-31bc-4420-900e-f5efcdd851d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:54.583619 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-ed0d510e-31bc-4420-900e-f5efcdd851d5 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:54.585239 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-ed0d510e-31bc-4420-900e-f5efcdd851d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:54.649078 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:54.660174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 210/429] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:24:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 19:24:54.677417 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0155f0a-b114-4c78-bf64-cece3ad2a57d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:54.686611 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 220/430] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 19:24:54.711755 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0d510e-31bc-4420-900e-f5efcdd851d5 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:54.788705 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0155f0a-b114-4c78-bf64-cece3ad2a57d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:54.801750 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:54.824742 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0d510e-31bc-4420-900e-f5efcdd851d5 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:54.827709 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404dbd5b-63e8-4a04-b342-61e972590923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:54.828636 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404dbd5b-63e8-4a04-b342-61e972590923 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:54.828925 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404dbd5b-63e8-4a04-b342-61e972590923 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:54.829201 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404dbd5b-63e8-4a04-b342-61e972590923 None admin] PATH_INFO: `/v3/projects/c4f989e5fad14bcebae5e279f22ee21d` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:54.830439 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404dbd5b-63e8-4a04-b342-61e972590923 None admin] RBAC: Authorizing `identity:get_project(project_id=c4f989e5fad14bcebae5e279f22ee21d)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:54.831882 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404dbd5b-63e8-4a04-b342-61e972590923 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:54.834044 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 211/431] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:54 2022] GET /identity/v3/projects/c4f989e5fad14bcebae5e279f22ee21d => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:24:54.840076 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404dbd5b-63e8-4a04-b342-61e972590923 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:54.862466 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52598e4f-b6cc-4a28-bcbc-5f1e49f9ede2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:54.863404 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52598e4f-b6cc-4a28-bcbc-5f1e49f9ede2 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:54.863705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52598e4f-b6cc-4a28-bcbc-5f1e49f9ede2 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:54.864052 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52598e4f-b6cc-4a28-bcbc-5f1e49f9ede2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:54.875514 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52598e4f-b6cc-4a28-bcbc-5f1e49f9ede2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:54.886976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-52598e4f-b6cc-4a28-bcbc-5f1e49f9ede2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 19:24:54.886976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:54.886976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:54.886976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:54.886976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:54.886976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:54.886976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:54.886976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:54.886976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:54.886976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:54.886976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:54.886976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:54.886976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:54.886976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:54.886976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:54.886976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:54.887873 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:54.887873 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:54.887873 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:54.887873 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:54.887873 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:54.887873 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:54.887873 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:54.887873 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:54.887873 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:54.887873 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:54.887873 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:54.887873 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:54.887873 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:54.887873 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:54.887873 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:54.887873 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:54.889093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:54.889093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:54.889093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:54.889093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:54.889093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:54.889093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:54.889093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:54.889093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:54.889093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:54.889093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:54.889093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 19:24:54.889093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:24:54.889093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 221/432] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jun 15 19:24:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:24:54.894115 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52598e4f-b6cc-4a28-bcbc-5f1e49f9ede2 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:54.935857 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:54.964214 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404dbd5b-63e8-4a04-b342-61e972590923 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:55.013048 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983c61e-94fc-4321-9c97-a85d8f052e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:55.015232 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983c61e-94fc-4321-9c97-a85d8f052e9e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:55.015523 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983c61e-94fc-4321-9c97-a85d8f052e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:55.015804 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983c61e-94fc-4321-9c97-a85d8f052e9e None admin] PATH_INFO: `/v3/groups/4fd7c2055c3147f5ad7054d54f3a1ad2` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:55.017451 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3983c61e-94fc-4321-9c97-a85d8f052e9e None admin] RBAC: Authorizing `identity:get_group(group_id=4fd7c2055c3147f5ad7054d54f3a1ad2)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:55.018539 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3983c61e-94fc-4321-9c97-a85d8f052e9e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:55.020655 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 212/433] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Jun 15 19:24:54 2022] GET /identity/v3/groups/4fd7c2055c3147f5ad7054d54f3a1ad2 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:55.023469 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52598e4f-b6cc-4a28-bcbc-5f1e49f9ede2 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:55.030020 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983c61e-94fc-4321-9c97-a85d8f052e9e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:55.062601 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4eab14-2cc7-431e-a770-068612af3ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:55.063672 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4eab14-2cc7-431e-a770-068612af3ecd None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:55.064198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4eab14-2cc7-431e-a770-068612af3ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:55.064691 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4eab14-2cc7-431e-a770-068612af3ecd None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:55.066057 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4eab14-2cc7-431e-a770-068612af3ecd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:55.066606 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4eab14-2cc7-431e-a770-068612af3ecd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:55.068228 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4eab14-2cc7-431e-a770-068612af3ecd None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:55.078381 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 222/434] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:24:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:24:55.084325 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4eab14-2cc7-431e-a770-068612af3ecd None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:55.085043 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:55.145027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983c61e-94fc-4321-9c97-a85d8f052e9e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:55.181356 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2241023-f269-4151-930d-52f29b269ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:55.181847 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2241023-f269-4151-930d-52f29b269ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:55.182161 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2241023-f269-4151-930d-52f29b269ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:55.182442 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2241023-f269-4151-930d-52f29b269ab7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:55.183330 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2241023-f269-4151-930d-52f29b269ab7 None admin] RBAC: Adding query filter params (group.id=4fd7c2055c3147f5ad7054d54f3a1ad2, role.id=3920d5126e22480ab9a4eec9cf0fd4e7, scope.project.id=c4f989e5fad14bcebae5e279f22ee21d) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:55.183719 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2241023-f269-4151-930d-52f29b269ab7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:55.185099 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2241023-f269-4151-930d-52f29b269ab7 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:55.192280 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 213/435] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Jun 15 19:24:55 2022] GET /identity/v3/role_assignments?group.id=4fd7c2055c3147f5ad7054d54f3a1ad2&scope.project.id=c4f989e5fad14bcebae5e279f22ee21d&role.id=3920d5126e22480ab9a4eec9cf0fd4e7 => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:55.196923 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:55.204892 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4eab14-2cc7-431e-a770-068612af3ecd None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:55.246050 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a9b1aa-de72-4b9b-b646-5217c21ef977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:55.247098 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9b1aa-de72-4b9b-b646-5217c21ef977 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:55.247583 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9b1aa-de72-4b9b-b646-5217c21ef977 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:55.248083 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9b1aa-de72-4b9b-b646-5217c21ef977 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:55.326003 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a9b1aa-de72-4b9b-b646-5217c21ef977 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:55.327873 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a9b1aa-de72-4b9b-b646-5217c21ef977 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:55.432542 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:55.439368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 223/436] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:24:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 19:24:55.444959 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2241023-f269-4151-930d-52f29b269ab7 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:55.554284 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2241023-f269-4151-930d-52f29b269ab7 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:55.567956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:55.596159 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da19c03-e662-40d6-b975-8039b8928f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:55.597027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da19c03-e662-40d6-b975-8039b8928f3e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:55.597305 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da19c03-e662-40d6-b975-8039b8928f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:55.597579 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da19c03-e662-40d6-b975-8039b8928f3e None admin] PATH_INFO: `/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:55.598944 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da19c03-e662-40d6-b975-8039b8928f3e None admin] RBAC: Authorizing `identity:get_project(project_id=383a1a7fd67a4b0c8baf1403a8b02f25)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:55.600382 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da19c03-e662-40d6-b975-8039b8928f3e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:55.602419 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 214/437] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:55 2022] GET /identity/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:24:55.608102 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a9b1aa-de72-4b9b-b646-5217c21ef977 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:55.729062 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a9b1aa-de72-4b9b-b646-5217c21ef977 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:55.763616 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04057f-93e4-46a6-967e-ba4a010bee4c None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:55.763801 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f600fb-b8ef-4fc1-96f3-2c9d32960376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:55.763908 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04057f-93e4-46a6-967e-ba4a010bee4c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:55.764157 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04057f-93e4-46a6-967e-ba4a010bee4c None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:55.764570 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f600fb-b8ef-4fc1-96f3-2c9d32960376 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:55.764850 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f600fb-b8ef-4fc1-96f3-2c9d32960376 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:55.765119 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f600fb-b8ef-4fc1-96f3-2c9d32960376 None admin] PATH_INFO: `/v3/groups/4fd7c2055c3147f5ad7054d54f3a1ad2` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:55.765532 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 215/438] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 19:24:55.766433 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f600fb-b8ef-4fc1-96f3-2c9d32960376 None admin] RBAC: Authorizing `identity:get_group(group_id=4fd7c2055c3147f5ad7054d54f3a1ad2)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:55.766634 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b6a09-6033-4970-950d-72208b276653 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:55.766919 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b6a09-6033-4970-950d-72208b276653 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:55.767246 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b6a09-6033-4970-950d-72208b276653 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:55.767939 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f600fb-b8ef-4fc1-96f3-2c9d32960376 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:55.768525 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 216/439] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:55.769676 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 224/440] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Jun 15 19:24:55 2022] GET /identity/v3/groups/4fd7c2055c3147f5ad7054d54f3a1ad2 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:55.771799 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4584f145-0169-4e1d-9f56-a3fcfe81d688 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:55.772149 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4584f145-0169-4e1d-9f56-a3fcfe81d688 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:55.772488 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4584f145-0169-4e1d-9f56-a3fcfe81d688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:55.775082 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f600fb-b8ef-4fc1-96f3-2c9d32960376 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:55.789846 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-4584f145-0169-4e1d-9f56-a3fcfe81d688 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:55.791073 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-4584f145-0169-4e1d-9f56-a3fcfe81d688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:55.835753 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:55.857047 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 217/441] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 19:24:55.861806 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8812d-4c4d-4e50-94ef-5ab81db08e82 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:55.862117 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8812d-4c4d-4e50-94ef-5ab81db08e82 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:55.862406 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8812d-4c4d-4e50-94ef-5ab81db08e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:55.874007 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f600fb-b8ef-4fc1-96f3-2c9d32960376 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:55.878026 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-a8e8812d-4c4d-4e50-94ef-5ab81db08e82 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:24:55.879305 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a8e8812d-4c4d-4e50-94ef-5ab81db08e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:24:55.908878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1893ef7-0833-492d-b867-5ff43f9081f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:55.909526 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1893ef7-0833-492d-b867-5ff43f9081f2 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:55.909810 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1893ef7-0833-492d-b867-5ff43f9081f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:55.910110 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1893ef7-0833-492d-b867-5ff43f9081f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:55.910935 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1893ef7-0833-492d-b867-5ff43f9081f2 None admin] RBAC: Adding query filter params (group.id=4fd7c2055c3147f5ad7054d54f3a1ad2, role.id=3920d5126e22480ab9a4eec9cf0fd4e7, scope.project.id=383a1a7fd67a4b0c8baf1403a8b02f25) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:55.911241 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1893ef7-0833-492d-b867-5ff43f9081f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:55.912705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1893ef7-0833-492d-b867-5ff43f9081f2 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:55.919116 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 225/442] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Jun 15 19:24:55 2022] GET /identity/v3/role_assignments?group.id=4fd7c2055c3147f5ad7054d54f3a1ad2&scope.project.id=383a1a7fd67a4b0c8baf1403a8b02f25&role.id=3920d5126e22480ab9a4eec9cf0fd4e7 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:24:55.922405 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1893ef7-0833-492d-b867-5ff43f9081f2 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:55.949447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 218/443] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:24:55.985888 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e8812d-4c4d-4e50-94ef-5ab81db08e82 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:56.039500 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:56.042133 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1893ef7-0833-492d-b867-5ff43f9081f2 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:56.072306 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:56.081348 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1d9d0-cecf-4175-b4c7-88445eac56c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:56.082200 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1d9d0-cecf-4175-b4c7-88445eac56c7 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:56.082485 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1d9d0-cecf-4175-b4c7-88445eac56c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:56.082762 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1d9d0-cecf-4175-b4c7-88445eac56c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:56.094974 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc1d9d0-cecf-4175-b4c7-88445eac56c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:56.102820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-0fc1d9d0-cecf-4175-b4c7-88445eac56c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:24:56.102820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:56.102820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:56.102820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:56.102820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:56.102820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:56.102820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:56.102820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:56.102820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:56.102820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:56.102820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:56.102820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:56.102820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:56.102820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:56.102820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:56.102820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:56.104569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:56.104569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:56.104569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:56.104569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:56.104569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:56.104569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:56.104569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:56.104569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:56.104569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:56.104569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:56.104569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:56.104569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:56.104569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:56.104569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:56.104569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:56.104569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:56.106346 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:56.106346 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:56.106346 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:56.106346 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:56.106346 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:56.106346 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:56.106346 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:56.106346 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:56.106346 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:56.106346 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:56.106346 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:24:56.106346 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:24:56.110451 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 226/444] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jun 15 19:24:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:56.110565 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1d9d0-cecf-4175-b4c7-88445eac56c7 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:56.113080 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e8812d-4c4d-4e50-94ef-5ab81db08e82 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:56.154923 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a83ac-5bc2-4380-8363-bb3db0a9cc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:56.155738 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a83ac-5bc2-4380-8363-bb3db0a9cc99 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:56.156007 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a83ac-5bc2-4380-8363-bb3db0a9cc99 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:56.156283 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a83ac-5bc2-4380-8363-bb3db0a9cc99 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:56.169440 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2a83ac-5bc2-4380-8363-bb3db0a9cc99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:24:56.177307 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-8e2a83ac-5bc2-4380-8363-bb3db0a9cc99 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 19:24:56.177307 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:24:56.177307 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:24:56.177307 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:24:56.177307 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:24:56.177307 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:24:56.177307 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:24:56.177307 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:24:56.177307 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:24:56.177307 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:24:56.177307 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:24:56.177307 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:24:56.177307 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:24:56.177307 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:24:56.177307 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:24:56.177307 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:24:56.178208 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:24:56.178208 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:24:56.178208 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:24:56.178208 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:24:56.178208 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:24:56.178208 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:24:56.178208 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:24:56.178208 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:24:56.178208 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:24:56.178208 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:24:56.178208 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:24:56.178208 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:24:56.178208 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:24:56.178208 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:24:56.178208 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:24:56.178208 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:24:56.179029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:24:56.179029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:24:56.179029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:24:56.179029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:24:56.179029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:24:56.179029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:24:56.179029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:24:56.179029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:24:56.179029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:24:56.179029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:24:56.179029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 19:24:56.179029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:24:56.179029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 219/445] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:24:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:24:56.185907 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a83ac-5bc2-4380-8363-bb3db0a9cc99 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:56.221877 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1d9d0-cecf-4175-b4c7-88445eac56c7 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:56.255105 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923fe7f9-4de5-467a-bcb2-f04a3b61c9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:56.255767 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923fe7f9-4de5-467a-bcb2-f04a3b61c9f4 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:56.256053 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923fe7f9-4de5-467a-bcb2-f04a3b61c9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:56.256328 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923fe7f9-4de5-467a-bcb2-f04a3b61c9f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:56.257074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923fe7f9-4de5-467a-bcb2-f04a3b61c9f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:56.257380 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923fe7f9-4de5-467a-bcb2-f04a3b61c9f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:56.259041 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923fe7f9-4de5-467a-bcb2-f04a3b61c9f4 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:56.267192 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 227/446] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:24:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:24:56.272530 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923fe7f9-4de5-467a-bcb2-f04a3b61c9f4 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:56.281802 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:56.285610 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a83ac-5bc2-4380-8363-bb3db0a9cc99 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:56.318893 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8288dcb-8a66-4ccc-ab29-6ffacc94e91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:56.319550 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8288dcb-8a66-4ccc-ab29-6ffacc94e91e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:56.319829 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8288dcb-8a66-4ccc-ab29-6ffacc94e91e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:56.320103 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8288dcb-8a66-4ccc-ab29-6ffacc94e91e None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:56.320843 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8288dcb-8a66-4ccc-ab29-6ffacc94e91e None admin] RBAC: Adding query filter params (name=member) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:56.321143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8288dcb-8a66-4ccc-ab29-6ffacc94e91e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:56.322533 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8288dcb-8a66-4ccc-ab29-6ffacc94e91e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:56.323403 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:56.333337 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 220/447] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jun 15 19:24:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:24:56.338934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8288dcb-8a66-4ccc-ab29-6ffacc94e91e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:56.373955 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923fe7f9-4de5-467a-bcb2-f04a3b61c9f4 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:56.407150 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4af63-15ca-4b7d-88ab-76721a0ea435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:56.407790 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4af63-15ca-4b7d-88ab-76721a0ea435 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:56.408073 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4af63-15ca-4b7d-88ab-76721a0ea435 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:56.408348 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4af63-15ca-4b7d-88ab-76721a0ea435 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:56.437572 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8288dcb-8a66-4ccc-ab29-6ffacc94e91e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:56.469821 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e878289-a81e-48c9-bb3e-60c386fce977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:56.470483 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e878289-a81e-48c9-bb3e-60c386fce977 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:56.470768 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e878289-a81e-48c9-bb3e-60c386fce977 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:56.471041 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e878289-a81e-48c9-bb3e-60c386fce977 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:56.474533 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d4af63-15ca-4b7d-88ab-76721a0ea435 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:56.475983 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d4af63-15ca-4b7d-88ab-76721a0ea435 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:56.498053 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:56.524945 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:56.537123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e878289-a81e-48c9-bb3e-60c386fce977 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:56.538605 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e878289-a81e-48c9-bb3e-60c386fce977 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:56.575001 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 228/448] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:24:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 19:24:56.580490 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4af63-15ca-4b7d-88ab-76721a0ea435 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:56.637522 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 221/449] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:24:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 19:24:56.643071 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e878289-a81e-48c9-bb3e-60c386fce977 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:56.676637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4af63-15ca-4b7d-88ab-76721a0ea435 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:56.698283 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:56.710344 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d371898-35a5-425f-a5af-dd3cbe5604d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:56.711200 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d371898-35a5-425f-a5af-dd3cbe5604d3 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:56.711488 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d371898-35a5-425f-a5af-dd3cbe5604d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:56.711767 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d371898-35a5-425f-a5af-dd3cbe5604d3 None admin] PATH_INFO: `/v3/projects/7d07be90a27c40718a346a1d9e5469b4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:56.712951 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d371898-35a5-425f-a5af-dd3cbe5604d3 None admin] RBAC: Authorizing `identity:get_project(project_id=7d07be90a27c40718a346a1d9e5469b4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:56.714960 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d371898-35a5-425f-a5af-dd3cbe5604d3 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:56.716659 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 229/450] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:56 2022] GET /identity/v3/projects/7d07be90a27c40718a346a1d9e5469b4 => generated 344 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:24:56.721927 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d371898-35a5-425f-a5af-dd3cbe5604d3 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:56.738444 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e878289-a81e-48c9-bb3e-60c386fce977 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:56.739681 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:56.773202 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b104c26-b398-4700-881b-8d8e64bc9405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:56.774052 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b104c26-b398-4700-881b-8d8e64bc9405 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:56.774363 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b104c26-b398-4700-881b-8d8e64bc9405 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:56.774641 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b104c26-b398-4700-881b-8d8e64bc9405 None admin] PATH_INFO: `/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:56.775889 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b104c26-b398-4700-881b-8d8e64bc9405 None admin] RBAC: Authorizing `identity:get_project(project_id=383a1a7fd67a4b0c8baf1403a8b02f25)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:56.777311 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b104c26-b398-4700-881b-8d8e64bc9405 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:56.779027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 222/451] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:24:56 2022] GET /identity/v3/projects/383a1a7fd67a4b0c8baf1403a8b02f25 => generated 302 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:24:56.784323 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b104c26-b398-4700-881b-8d8e64bc9405 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:56.818956 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d371898-35a5-425f-a5af-dd3cbe5604d3 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:56.851818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98cf84-6bc6-4a55-a539-6ebd09a54cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:56.852663 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98cf84-6bc6-4a55-a539-6ebd09a54cec None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:56.852942 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98cf84-6bc6-4a55-a539-6ebd09a54cec None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:56.853210 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98cf84-6bc6-4a55-a539-6ebd09a54cec None admin] PATH_INFO: `/v3/groups/2f50a7a67b8e4e54b7289d1f6ba6a1c6` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:56.854395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d98cf84-6bc6-4a55-a539-6ebd09a54cec None admin] RBAC: Authorizing `identity:get_group(group_id=2f50a7a67b8e4e54b7289d1f6ba6a1c6)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:56.855846 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d98cf84-6bc6-4a55-a539-6ebd09a54cec None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:56.857442 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 230/452] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Jun 15 19:24:56 2022] GET /identity/v3/groups/2f50a7a67b8e4e54b7289d1f6ba6a1c6 => generated 229 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:24:56.863267 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98cf84-6bc6-4a55-a539-6ebd09a54cec None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:56.881453 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b104c26-b398-4700-881b-8d8e64bc9405 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:56.907563 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:56.915521 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8d551-7ab7-482f-92ab-8a25d189363e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:56.916360 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8d551-7ab7-482f-92ab-8a25d189363e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:56.916635 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8d551-7ab7-482f-92ab-8a25d189363e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:56.916903 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8d551-7ab7-482f-92ab-8a25d189363e None admin] PATH_INFO: `/v3/groups/4fd7c2055c3147f5ad7054d54f3a1ad2` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:56.918118 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de8d551-7ab7-482f-92ab-8a25d189363e None admin] RBAC: Authorizing `identity:get_group(group_id=4fd7c2055c3147f5ad7054d54f3a1ad2)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:56.919563 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de8d551-7ab7-482f-92ab-8a25d189363e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:56.921160 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 223/453] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Jun 15 19:24:56 2022] GET /identity/v3/groups/4fd7c2055c3147f5ad7054d54f3a1ad2 => generated 226 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:24:56.927348 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8d551-7ab7-482f-92ab-8a25d189363e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:24:56.947272 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:24:56.962003 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98cf84-6bc6-4a55-a539-6ebd09a54cec None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:56.998669 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee3a50d-ac83-46c7-b8f2-a1b59e1d43e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:57.002392 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3a50d-ac83-46c7-b8f2-a1b59e1d43e6 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:57.002392 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3a50d-ac83-46c7-b8f2-a1b59e1d43e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:57.002392 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3a50d-ac83-46c7-b8f2-a1b59e1d43e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:57.002392 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee3a50d-ac83-46c7-b8f2-a1b59e1d43e6 None admin] RBAC: Adding query filter params (group.id=2f50a7a67b8e4e54b7289d1f6ba6a1c6, role.id=4f736313c26d4f8db48bbb2f346e50e4, scope.project.id=7d07be90a27c40718a346a1d9e5469b4) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:57.002392 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee3a50d-ac83-46c7-b8f2-a1b59e1d43e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:57.002989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee3a50d-ac83-46c7-b8f2-a1b59e1d43e6 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:57.014197 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 231/454] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Jun 15 19:24:56 2022] GET /identity/v3/role_assignments?group.id=2f50a7a67b8e4e54b7289d1f6ba6a1c6&scope.project.id=7d07be90a27c40718a346a1d9e5469b4&role.id=4f736313c26d4f8db48bbb2f346e50e4 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:24:57.037468 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8d551-7ab7-482f-92ab-8a25d189363e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:24:57.074784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee7db7-2dfc-4c0a-b691-fe4b7110f1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:24:57.075449 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee7db7-2dfc-4c0a-b691-fe4b7110f1ca None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:57.075730 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee7db7-2dfc-4c0a-b691-fe4b7110f1ca None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:57.076002 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee7db7-2dfc-4c0a-b691-fe4b7110f1ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:57.076852 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aee7db7-2dfc-4c0a-b691-fe4b7110f1ca None admin] RBAC: Adding query filter params (group.id=4fd7c2055c3147f5ad7054d54f3a1ad2, role.id=e53a78912e764f8387e0cf637bb86bee, scope.project.id=383a1a7fd67a4b0c8baf1403a8b02f25) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:24:57.077167 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aee7db7-2dfc-4c0a-b691-fe4b7110f1ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:24:57.078666 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aee7db7-2dfc-4c0a-b691-fe4b7110f1ca None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:24:57.085389 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 224/455] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Jun 15 19:24:56 2022] GET /identity/v3/role_assignments?group.id=4fd7c2055c3147f5ad7054d54f3a1ad2&scope.project.id=383a1a7fd67a4b0c8baf1403a8b02f25&role.id=e53a78912e764f8387e0cf637bb86bee => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:24:59.970702 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b600c0-6a96-455e-b0f5-5a8bd6c0d929 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:59.971216 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b600c0-6a96-455e-b0f5-5a8bd6c0d929 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:59.971321 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b600c0-6a96-455e-b0f5-5a8bd6c0d929 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:24:59.972856 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 232/456] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:24:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:24:59.978573 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86244f15-d062-4ab5-a25c-6fc6aafe1664 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:24:59.979416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86244f15-d062-4ab5-a25c-6fc6aafe1664 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:24:59.980001 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86244f15-d062-4ab5-a25c-6fc6aafe1664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:00.007109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-86244f15-d062-4ab5-a25c-6fc6aafe1664 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:00.008909 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-86244f15-d062-4ab5-a25c-6fc6aafe1664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:00.079914 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:00.096697 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 225/457] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:24:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 19:25:00.138249 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b600c0-6a96-455e-b0f5-5a8bd6c0d929 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:00.276900 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f80fdc-8a61-49fa-8b9e-1a0edc025bee None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:00.277634 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f80fdc-8a61-49fa-8b9e-1a0edc025bee None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:00.278155 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f80fdc-8a61-49fa-8b9e-1a0edc025bee None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:00.279901 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b600c0-6a96-455e-b0f5-5a8bd6c0d929 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:00.281512 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 226/458] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:00.286391 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353215f-c850-45b3-83ed-8814d1401192 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:00.286682 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353215f-c850-45b3-83ed-8814d1401192 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:00.286966 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353215f-c850-45b3-83ed-8814d1401192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:00.318966 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c353215f-c850-45b3-83ed-8814d1401192 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:00.320167 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c353215f-c850-45b3-83ed-8814d1401192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:00.344395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f5f6d-123a-4de6-9ccf-2990f721d822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:00.345209 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f5f6d-123a-4de6-9ccf-2990f721d822 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:00.345491 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f5f6d-123a-4de6-9ccf-2990f721d822 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:00.345770 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f5f6d-123a-4de6-9ccf-2990f721d822 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:00.357281 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f5f6d-123a-4de6-9ccf-2990f721d822 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:00.358934 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f5f6d-123a-4de6-9ccf-2990f721d822 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:00.377424 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-120f5f6d-123a-4de6-9ccf-2990f721d822 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:00.377424 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:00.377424 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:00.377424 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:00.377424 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:00.377424 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:00.377424 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:00.377424 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:00.377424 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:00.377424 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:00.377424 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:00.377424 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:00.377424 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:00.377424 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:00.377424 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:00.378379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:00.378379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:00.378379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:00.378379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:00.378379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:00.378379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:00.378379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:00.378379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:00.378379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:00.378379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:00.378379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:00.378379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:00.378379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:00.378379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:00.378379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:00.378379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:00.378379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:00.379730 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:00.379730 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:00.379730 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:00.379730 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:00.379730 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 233/459] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:00.380322 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf163e58-83b0-450a-aa51-be1c945a3fae None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:00.385479 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf163e58-83b0-450a-aa51-be1c945a3fae None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:00.386455 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf163e58-83b0-450a-aa51-be1c945a3fae None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:00.388108 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 234/460] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:00.389184 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf163e58-83b0-450a-aa51-be1c945a3fae None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:00.393205 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:00.409378 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 227/461] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:25:00.410984 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f252c-dcb7-4910-b29e-2a9c5e616717 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:00.411277 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f252c-dcb7-4910-b29e-2a9c5e616717 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:00.411552 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f252c-dcb7-4910-b29e-2a9c5e616717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:00.428059 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-ef0f252c-dcb7-4910-b29e-2a9c5e616717 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:00.429246 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ef0f252c-dcb7-4910-b29e-2a9c5e616717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:00.500428 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf163e58-83b0-450a-aa51-be1c945a3fae None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:00.530268 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 228/462] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:25:00.531166 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f252c-dcb7-4910-b29e-2a9c5e616717 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:00.544946 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e330649-cf53-44d6-b2d2-1e64d9e32d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:00.545671 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e330649-cf53-44d6-b2d2-1e64d9e32d79 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:00.545957 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e330649-cf53-44d6-b2d2-1e64d9e32d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:00.546255 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e330649-cf53-44d6-b2d2-1e64d9e32d79 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:00.547245 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e330649-cf53-44d6-b2d2-1e64d9e32d79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:00.547557 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e330649-cf53-44d6-b2d2-1e64d9e32d79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:00.550341 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e330649-cf53-44d6-b2d2-1e64d9e32d79 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:00.567645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 235/463] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:00.568476 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e330649-cf53-44d6-b2d2-1e64d9e32d79 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:00.596143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:00.643713 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:00.654167 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f252c-dcb7-4910-b29e-2a9c5e616717 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:00.687429 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e330649-cf53-44d6-b2d2-1e64d9e32d79 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:00.691067 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3bdbba-1a37-43ee-b5fd-71dc7e62cbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:00.692057 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3bdbba-1a37-43ee-b5fd-71dc7e62cbe9 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:00.692475 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3bdbba-1a37-43ee-b5fd-71dc7e62cbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:00.692887 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3bdbba-1a37-43ee-b5fd-71dc7e62cbe9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:00.704003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3bdbba-1a37-43ee-b5fd-71dc7e62cbe9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:00.705655 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3bdbba-1a37-43ee-b5fd-71dc7e62cbe9 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:00.720702 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a5aa66-b80e-4a20-915b-22395a6a1097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:00.721485 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5aa66-b80e-4a20-915b-22395a6a1097 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:00.721771 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5aa66-b80e-4a20-915b-22395a6a1097 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:00.722045 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5aa66-b80e-4a20-915b-22395a6a1097 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:00.726399 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-ee3bdbba-1a37-43ee-b5fd-71dc7e62cbe9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:00.726399 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:00.726399 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:00.726399 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:00.726399 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:00.726399 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:00.726399 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:00.726399 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:00.726399 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:00.726399 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:00.726399 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:00.726399 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:00.726399 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:00.726399 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:00.726399 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:00.727384 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:00.727384 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:00.727384 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:00.727384 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:00.727384 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:00.727384 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:00.727384 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:00.727384 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:00.727384 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:00.727384 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:00.727384 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:00.727384 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:00.727384 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:00.727384 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:00.727384 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:00.727384 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:00.727384 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:00.728438 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:00.728438 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:00.728438 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:00.728438 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:00.730951 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 229/464] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:00.731853 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3bdbba-1a37-43ee-b5fd-71dc7e62cbe9 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:00.737526 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a5aa66-b80e-4a20-915b-22395a6a1097 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:00.739010 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a5aa66-b80e-4a20-915b-22395a6a1097 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:00.756042 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-e8a5aa66-b80e-4a20-915b-22395a6a1097 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:00.756042 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:00.756042 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:00.756042 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:00.756042 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:00.756042 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:00.756042 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:00.756042 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:00.756042 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:00.756042 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:00.756042 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:00.756042 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:00.756042 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:00.756042 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:00.756042 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:00.756899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:00.756899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:00.756899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:00.756899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:00.756899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:00.756899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:00.756899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:00.756899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:00.756899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:00.756899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:00.756899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:00.756899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:00.756899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:00.756899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:00.756899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:00.756899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:00.756899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:00.757741 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:00.757741 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:00.757741 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:00.757741 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:00.762423 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 236/465] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:00.763918 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1fa9b3-21a0-4af8-89d0-5572cdd59523 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:00.764212 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1fa9b3-21a0-4af8-89d0-5572cdd59523 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:00.764532 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1fa9b3-21a0-4af8-89d0-5572cdd59523 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:00.765954 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 237/466] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:00.770470 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1fa9b3-21a0-4af8-89d0-5572cdd59523 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:00.836289 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3bdbba-1a37-43ee-b5fd-71dc7e62cbe9 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:00.838326 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:00.874582 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d1868-9e48-4cd5-b323-e8373eade7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:00.875251 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d1868-9e48-4cd5-b323-e8373eade7ae None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:00.875532 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d1868-9e48-4cd5-b323-e8373eade7ae None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:00.875630 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1fa9b3-21a0-4af8-89d0-5572cdd59523 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:00.875795 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d1868-9e48-4cd5-b323-e8373eade7ae None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:00.876498 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d1868-9e48-4cd5-b323-e8373eade7ae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:00.878508 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d1868-9e48-4cd5-b323-e8373eade7ae None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:00.892873 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:00.899999 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 230/467] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Jun 15 19:25:00 2022] POST /identity/v3/users => generated 246 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:25:00.900839 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d1868-9e48-4cd5-b323-e8373eade7ae None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:00.929949 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85eba8-8cd4-4876-94d1-e12213e5b08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:00.930704 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85eba8-8cd4-4876-94d1-e12213e5b08f None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:00.930989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85eba8-8cd4-4876-94d1-e12213e5b08f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:00.931314 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85eba8-8cd4-4876-94d1-e12213e5b08f None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:00.932060 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85eba8-8cd4-4876-94d1-e12213e5b08f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:00.932367 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85eba8-8cd4-4876-94d1-e12213e5b08f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:00.934959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85eba8-8cd4-4876-94d1-e12213e5b08f None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:00.952584 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 238/468] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:00.952584 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c433a43-0d4a-4aef-8c12-5e8e4e0a4523 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:00.952584 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c433a43-0d4a-4aef-8c12-5e8e4e0a4523 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:00.952584 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c433a43-0d4a-4aef-8c12-5e8e4e0a4523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:00.971446 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-8c433a43-0d4a-4aef-8c12-5e8e4e0a4523 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:00.972927 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-8c433a43-0d4a-4aef-8c12-5e8e4e0a4523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:01.058814 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 239/469] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:25:01.058814 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c433a43-0d4a-4aef-8c12-5e8e4e0a4523 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:01.070458 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d1868-9e48-4cd5-b323-e8373eade7ae None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:01.117870 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c40ad5-5c9a-4f00-b219-e2ca69be29da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:01.117870 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c40ad5-5c9a-4f00-b219-e2ca69be29da None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:01.118285 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c40ad5-5c9a-4f00-b219-e2ca69be29da None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:01.118462 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c40ad5-5c9a-4f00-b219-e2ca69be29da None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:01.119196 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c40ad5-5c9a-4f00-b219-e2ca69be29da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:01.119505 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c40ad5-5c9a-4f00-b219-e2ca69be29da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:01.120956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c40ad5-5c9a-4f00-b219-e2ca69be29da None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:01.136781 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 231/470] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:01.138070 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c40ad5-5c9a-4f00-b219-e2ca69be29da None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:01.162331 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:01.194731 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c433a43-0d4a-4aef-8c12-5e8e4e0a4523 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:01.206139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:01.237314 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db967fe0-766f-4dbb-8d42-ef0de43434ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:01.238418 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db967fe0-766f-4dbb-8d42-ef0de43434ca None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:01.238713 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db967fe0-766f-4dbb-8d42-ef0de43434ca None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:01.238994 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db967fe0-766f-4dbb-8d42-ef0de43434ca None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:01.239993 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db967fe0-766f-4dbb-8d42-ef0de43434ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:01.242308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db967fe0-766f-4dbb-8d42-ef0de43434ca None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:01.270212 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c40ad5-5c9a-4f00-b219-e2ca69be29da None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:01.270404 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 240/471] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Jun 15 19:25:01 2022] POST /identity/v3/users => generated 248 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:25:01.271254 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db967fe0-766f-4dbb-8d42-ef0de43434ca None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:01.345669 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970bd4f1-b80e-4e70-8284-15335906e961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:01.345669 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bd4f1-b80e-4e70-8284-15335906e961 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:01.345669 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bd4f1-b80e-4e70-8284-15335906e961 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:01.345669 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bd4f1-b80e-4e70-8284-15335906e961 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:01.369880 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970bd4f1-b80e-4e70-8284-15335906e961 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:01.372873 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970bd4f1-b80e-4e70-8284-15335906e961 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:01.408207 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-970bd4f1-b80e-4e70-8284-15335906e961 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:01.408207 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:01.408207 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:01.408207 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:01.408207 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:01.408207 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:01.408207 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:01.408207 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:01.408207 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:01.408207 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:01.408207 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:01.408207 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:01.408207 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:01.408207 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:01.408207 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:01.409143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:01.409143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:01.409143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:01.409143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:01.409143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:01.409143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:01.409143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:01.409143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:01.409143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:01.409143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:01.409143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:01.409143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:01.409143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:01.409143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:01.409143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:01.409143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:01.409143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:01.409982 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:01.409982 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:01.409982 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:01.409982 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:01.409982 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 232/472] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:01.416932 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970bd4f1-b80e-4e70-8284-15335906e961 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:01.505731 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db967fe0-766f-4dbb-8d42-ef0de43434ca None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:01.509398 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:01.545121 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae52d4-e3d7-4fc9-abe7-2f8595056962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:01.545793 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae52d4-e3d7-4fc9-abe7-2f8595056962 None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:01.546073 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae52d4-e3d7-4fc9-abe7-2f8595056962 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:01.546373 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae52d4-e3d7-4fc9-abe7-2f8595056962 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:01.546879 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970bd4f1-b80e-4e70-8284-15335906e961 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:01.547070 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae52d4-e3d7-4fc9-abe7-2f8595056962 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:01.548617 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae52d4-e3d7-4fc9-abe7-2f8595056962 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:01.572522 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 241/473] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Jun 15 19:25:01 2022] POST /identity/v3/users => generated 248 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:25:01.574940 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e747b5b-c3fa-43a2-950d-0b7c27d8e32d None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:01.575237 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e747b5b-c3fa-43a2-950d-0b7c27d8e32d None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:01.575519 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e747b5b-c3fa-43a2-950d-0b7c27d8e32d None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:01.581114 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 242/474] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:01.583107 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c385025-643b-406e-be4d-3f71570ef1ff None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:01.583475 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c385025-643b-406e-be4d-3f71570ef1ff None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:01.583833 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c385025-643b-406e-be4d-3f71570ef1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:01.600919 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4299d5-80b0-425a-b5c3-a0a1a1aed428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:01.601601 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4299d5-80b0-425a-b5c3-a0a1a1aed428 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:01.601883 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4299d5-80b0-425a-b5c3-a0a1a1aed428 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:01.602185 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4299d5-80b0-425a-b5c3-a0a1a1aed428 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:01.602956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4299d5-80b0-425a-b5c3-a0a1a1aed428 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:01.603218 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4299d5-80b0-425a-b5c3-a0a1a1aed428 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:01.603864 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-9c385025-643b-406e-be4d-3f71570ef1ff None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:01.604869 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4299d5-80b0-425a-b5c3-a0a1a1aed428 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:01.605182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-9c385025-643b-406e-be4d-3f71570ef1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:01.617617 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:01.621936 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 233/475] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:25:01.627686 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4299d5-80b0-425a-b5c3-a0a1a1aed428 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:01.696628 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 243/476] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:25:01.737017 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c385025-643b-406e-be4d-3f71570ef1ff None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:01.776673 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4299d5-80b0-425a-b5c3-a0a1a1aed428 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:01.815971 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0284a49d-6cbc-4150-8a75-7fda6252e5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:01.816657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284a49d-6cbc-4150-8a75-7fda6252e5b0 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:01.816942 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284a49d-6cbc-4150-8a75-7fda6252e5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:01.817219 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284a49d-6cbc-4150-8a75-7fda6252e5b0 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:01.817949 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0284a49d-6cbc-4150-8a75-7fda6252e5b0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:01.819595 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0284a49d-6cbc-4150-8a75-7fda6252e5b0 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:01.824592 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:01.856236 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 234/477] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Jun 15 19:25:01 2022] POST /identity/v3/users => generated 249 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:25:01.901207 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c385025-643b-406e-be4d-3f71570ef1ff None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:01.943151 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6463b-09c0-4d1a-afdd-ec9b9f8bced9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:01.944134 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6463b-09c0-4d1a-afdd-ec9b9f8bced9 None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:01.944556 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6463b-09c0-4d1a-afdd-ec9b9f8bced9 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:01.944966 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6463b-09c0-4d1a-afdd-ec9b9f8bced9 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:01.945877 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba6463b-09c0-4d1a-afdd-ec9b9f8bced9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:01.947444 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba6463b-09c0-4d1a-afdd-ec9b9f8bced9 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:01.955625 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.sql.core [None req-1ba6463b-09c0-4d1a-afdd-ec9b9f8bced9 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 15 19:25:01.955625 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 15 19:25:01.955625 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [parameters: {'id': 'e4e1e12eb1f1494cbd60a58af540f03c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 15 19:25:01.955625 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80223) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 15 19:25:01.974420 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-1ba6463b-09c0-4d1a-afdd-ec9b9f8bced9 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 15 19:25:01.974420 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:01.974420 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 19:25:01.974420 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 19:25:01.974420 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 19:25:01.974420 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 19:25:01.974420 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 19:25:01.974420 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 19:25:01.974420 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 19:25:01.974420 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application conn.query(q) Jun 15 19:25:01.974420 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 19:25:01.974420 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 19:25:01.974420 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 19:25:01.974420 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application result.read() Jun 15 19:25:01.974420 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 19:25:01.975976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 19:25:01.975976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 19:25:01.975976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 19:25:01.975976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 19:25:01.975976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 19:25:01.975976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 19:25:01.975976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 19:25:01.975976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 15 19:25:01.975976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:01.975976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 15 19:25:01.975976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:01.975976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:01.975976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 15 19:25:01.975976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 15 19:25:01.975976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 15 19:25:01.975976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return ref.to_dict() Jun 15 19:25:01.975976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 19:25:01.975976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application next(self.gen) Jun 15 19:25:01.977094 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 15 19:25:01.977094 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application yield resource Jun 15 19:25:01.977094 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 19:25:01.977094 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application next(self.gen) Jun 15 19:25:01.977094 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 15 19:25:01.977094 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self.session.rollback() Jun 15 19:25:01.977094 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 15 19:25:01.977094 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self.force_reraise() Jun 15 19:25:01.977094 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 15 19:25:01.977094 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise self.value Jun 15 19:25:01.977094 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 15 19:25:01.977094 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 15 19:25:01.977094 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 15 19:25:01.977094 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application session.commit() Jun 15 19:25:01.977094 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 15 19:25:01.977094 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 15 19:25:01.977094 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 15 19:25:01.978246 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._prepare_impl() Jun 15 19:25:01.978246 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 15 19:25:01.978246 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self.session.flush() Jun 15 19:25:01.978246 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 15 19:25:01.978246 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._flush(objects) Jun 15 19:25:01.978246 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 15 19:25:01.978246 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 15 19:25:01.978246 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 15 19:25:01.978246 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application compat.raise_( Jun 15 19:25:01.978246 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 19:25:01.978246 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception Jun 15 19:25:01.978246 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 15 19:25:01.978246 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application flush_context.execute() Jun 15 19:25:01.978246 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 15 19:25:01.978246 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rec.execute(self) Jun 15 19:25:01.978246 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 15 19:25:01.978246 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 15 19:25:01.979144 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 15 19:25:01.979144 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 15 19:25:01.979144 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 15 19:25:01.979144 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 15 19:25:01.979144 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 15 19:25:01.979144 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 15 19:25:01.979144 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 15 19:25:01.979144 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 15 19:25:01.979144 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 15 19:25:01.979144 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 15 19:25:01.979144 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 15 19:25:01.979144 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 15 19:25:01.979144 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 15 19:25:01.979144 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 15 19:25:01.979144 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 19:25:01.979144 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception Jun 15 19:25:01.980517 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 19:25:01.980517 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 19:25:01.980517 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 19:25:01.980517 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 19:25:01.980517 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 19:25:01.980517 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 19:25:01.980517 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 19:25:01.980517 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application conn.query(q) Jun 15 19:25:01.980517 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 19:25:01.980517 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 19:25:01.980517 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 19:25:01.980517 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application result.read() Jun 15 19:25:01.980517 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 19:25:01.980517 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 19:25:01.980517 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 19:25:01.980517 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 19:25:01.981295 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 19:25:01.981295 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 19:25:01.981295 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 19:25:01.981295 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 19:25:01.981295 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 15 19:25:01.981295 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 15 19:25:01.981295 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [parameters: {'id': 'e4e1e12eb1f1494cbd60a58af540f03c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 15 19:25:01.981295 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 15 19:25:01.981295 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:01.981295 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 19:25:01.981295 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:01.981295 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:01.981295 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:01.981295 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:01.981295 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:01.981295 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:01.982287 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:01.982287 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:01.982287 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:01.982287 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:01.982287 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:01.982287 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:01.982287 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 15 19:25:01.982287 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 15 19:25:01.982287 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:01.982287 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:01.982287 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 15 19:25:01.982287 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 15 19:25:01.982287 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 15 19:25:01.982287 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 15 19:25:01.982287 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 15 19:25:01.982287 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:01.985097 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 244/478] 10.222.0.15 () {60 vars in 1131 bytes} [Wed Jun 15 19:25:01 2022] POST /identity/v3/roles => generated 144 bytes in 248 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:25:01.991022 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0284a49d-6cbc-4150-8a75-7fda6252e5b0 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:02.101804 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0284a49d-6cbc-4150-8a75-7fda6252e5b0 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:02.136497 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96657836-fa7c-4bbe-9f07-cee25700a122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:02.137536 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96657836-fa7c-4bbe-9f07-cee25700a122 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:02.138000 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96657836-fa7c-4bbe-9f07-cee25700a122 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:02.138435 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96657836-fa7c-4bbe-9f07-cee25700a122 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:02.148834 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96657836-fa7c-4bbe-9f07-cee25700a122 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:25:02.157669 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-96657836-fa7c-4bbe-9f07-cee25700a122 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 19:25:02.157669 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:02.157669 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:02.157669 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:02.157669 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:02.157669 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:02.157669 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:02.157669 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:02.157669 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:02.157669 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:02.157669 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:02.157669 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:02.157669 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:25:02.157669 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:25:02.157669 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:25:02.157669 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:25:02.158790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:25:02.158790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:25:02.158790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:25:02.158790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:25:02.158790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:02.158790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:02.158790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:02.158790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:02.158790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:02.158790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:02.158790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:02.158790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:02.158790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:02.158790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:02.158790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:02.158790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:02.159773 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:02.159773 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:02.159773 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:02.159773 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:02.159773 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:25:02.159773 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:25:02.159773 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:25:02.159773 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:25:02.159773 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:25:02.159773 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:25:02.159773 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 19:25:02.159773 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:02.162120 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 235/479] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jun 15 19:25:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:02.167573 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6463b-09c0-4d1a-afdd-ec9b9f8bced9 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:02.182465 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:02.275803 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6463b-09c0-4d1a-afdd-ec9b9f8bced9 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:02.311293 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f0cb45-d9fc-4f8d-8f7c-11e58ec41de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:02.311969 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0cb45-d9fc-4f8d-8f7c-11e58ec41de2 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:02.312250 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0cb45-d9fc-4f8d-8f7c-11e58ec41de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:02.312525 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0cb45-d9fc-4f8d-8f7c-11e58ec41de2 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:02.313287 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f0cb45-d9fc-4f8d-8f7c-11e58ec41de2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:02.313590 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f0cb45-d9fc-4f8d-8f7c-11e58ec41de2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:02.314990 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f0cb45-d9fc-4f8d-8f7c-11e58ec41de2 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:02.324155 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 245/480] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:25:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:03.417618 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08f90c-a9da-4d2d-b771-9d7d4b0257ee None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:03.418143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08f90c-a9da-4d2d-b771-9d7d4b0257ee None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:03.418256 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08f90c-a9da-4d2d-b771-9d7d4b0257ee None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:03.419842 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 236/481] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:03.440095 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48af19-7496-4b46-a2fb-b579e5bedf6c None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:03.440985 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48af19-7496-4b46-a2fb-b579e5bedf6c None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:03.441457 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48af19-7496-4b46-a2fb-b579e5bedf6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:03.461682 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-ca48af19-7496-4b46-a2fb-b579e5bedf6c None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:03.463566 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-ca48af19-7496-4b46-a2fb-b579e5bedf6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:03.556654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 246/482] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:25:03.604716 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da08f90c-a9da-4d2d-b771-9d7d4b0257ee None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:03.661230 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:03.742805 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da08f90c-a9da-4d2d-b771-9d7d4b0257ee None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:03.788170 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed70373-3ffc-44e5-bb98-1a0d18f40cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:03.788976 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed70373-3ffc-44e5-bb98-1a0d18f40cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:03.789265 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed70373-3ffc-44e5-bb98-1a0d18f40cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:03.789545 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed70373-3ffc-44e5-bb98-1a0d18f40cc7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:03.800159 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed70373-3ffc-44e5-bb98-1a0d18f40cc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:25:03.808466 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-bed70373-3ffc-44e5-bb98-1a0d18f40cc7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:25:03.808466 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:03.808466 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:03.808466 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:03.808466 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:03.808466 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:03.808466 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:03.808466 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:03.808466 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:03.808466 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:03.808466 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:03.808466 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:03.808466 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:25:03.808466 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:25:03.808466 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:25:03.808466 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:25:03.809332 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:25:03.809332 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:25:03.809332 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:25:03.809332 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:25:03.809332 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:03.809332 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:03.809332 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:03.809332 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:03.809332 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:03.809332 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:03.809332 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:03.809332 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:03.809332 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:03.809332 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:03.809332 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:03.809332 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:03.810167 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:03.810167 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:03.810167 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:03.810167 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:03.810167 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:25:03.810167 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:25:03.810167 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:25:03.810167 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:25:03.810167 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:25:03.810167 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:25:03.810167 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:25:03.810167 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:03.813985 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 237/483] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jun 15 19:25:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:03.815951 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca48af19-7496-4b46-a2fb-b579e5bedf6c None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:03.818142 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:03.954432 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca48af19-7496-4b46-a2fb-b579e5bedf6c None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:04.000676 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ea508-0c1d-4bf0-a835-88d18cbd115b None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:04.000992 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ea508-0c1d-4bf0-a835-88d18cbd115b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:04.001244 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ea508-0c1d-4bf0-a835-88d18cbd115b None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:04.002777 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb4fe9d-145b-461c-96d2-903665b98a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:04.003449 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4fe9d-145b-461c-96d2-903665b98a52 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:04.003732 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4fe9d-145b-461c-96d2-903665b98a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:04.003960 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 238/484] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:04.004059 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4fe9d-145b-461c-96d2-903665b98a52 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:04.004751 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb4fe9d-145b-461c-96d2-903665b98a52 None admin] RBAC: Adding query filter params (name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:04.005059 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb4fe9d-145b-461c-96d2-903665b98a52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:04.006450 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb4fe9d-145b-461c-96d2-903665b98a52 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:04.007834 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f050f7c-4d65-452c-99d2-e6b984057cdf None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:04.008123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f050f7c-4d65-452c-99d2-e6b984057cdf None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:04.008403 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f050f7c-4d65-452c-99d2-e6b984057cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:04.021442 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 247/485] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jun 15 19:25:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:04.027715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb4fe9d-145b-461c-96d2-903665b98a52 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:04.035374 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-7f050f7c-4d65-452c-99d2-e6b984057cdf None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:04.036686 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7f050f7c-4d65-452c-99d2-e6b984057cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:04.146399 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 239/486] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 19:25:04.154161 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:04.189943 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f050f7c-4d65-452c-99d2-e6b984057cdf None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:04.240579 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb4fe9d-145b-461c-96d2-903665b98a52 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:04.243688 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:04.326906 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6c441-6973-4f6e-b58b-83fcb7912353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:04.327601 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f050f7c-4d65-452c-99d2-e6b984057cdf None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:04.327710 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6c441-6973-4f6e-b58b-83fcb7912353 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:04.330649 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6c441-6973-4f6e-b58b-83fcb7912353 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:04.330649 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6c441-6973-4f6e-b58b-83fcb7912353 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:04.367033 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b6c441-6973-4f6e-b58b-83fcb7912353 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:04.368544 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b6c441-6973-4f6e-b58b-83fcb7912353 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:04.385437 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ecfe28-ac68-4450-b9f0-d938b571ccaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:04.386643 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ecfe28-ac68-4450-b9f0-d938b571ccaa None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:04.387143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ecfe28-ac68-4450-b9f0-d938b571ccaa None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:04.387607 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ecfe28-ac68-4450-b9f0-d938b571ccaa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:04.404824 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-f8b6c441-6973-4f6e-b58b-83fcb7912353 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:04.404824 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:04.404824 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:04.404824 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:04.404824 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:04.404824 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:04.404824 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:04.404824 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:04.404824 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:04.404824 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:04.404824 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:04.404824 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:04.404824 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:04.404824 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:04.404824 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:04.405725 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:04.405725 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:04.405725 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:04.405725 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:04.405725 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:04.405725 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:04.405725 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:04.405725 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:04.405725 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:04.405725 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:04.405725 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:04.405725 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:04.405725 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:04.405725 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:04.405725 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:04.405725 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:04.405725 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:04.406592 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:04.406592 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:04.406592 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:04.406592 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:04.409517 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 248/487] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:25:04.411005 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de0c47-3ae3-404a-abeb-4bb4dcae83d6 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:04.411605 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de0c47-3ae3-404a-abeb-4bb4dcae83d6 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:04.412074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de0c47-3ae3-404a-abeb-4bb4dcae83d6 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:04.413525 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 249/488] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:04.414837 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de0c47-3ae3-404a-abeb-4bb4dcae83d6 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:04.415566 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ecfe28-ac68-4450-b9f0-d938b571ccaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:25:04.424032 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-b4ecfe28-ac68-4450-b9f0-d938b571ccaa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:25:04.424032 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:04.424032 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:04.424032 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:04.424032 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:04.424032 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:04.424032 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:04.424032 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:04.424032 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:04.424032 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:04.424032 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:04.424032 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:04.424032 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:25:04.424032 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:25:04.424032 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:25:04.424032 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:25:04.425312 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:25:04.425312 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:25:04.425312 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:25:04.425312 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:25:04.425312 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:04.425312 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:04.425312 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:04.425312 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:04.425312 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:04.425312 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:04.425312 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:04.425312 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:04.425312 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:04.425312 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:04.425312 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:04.425312 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:04.426140 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:04.426140 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:04.426140 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:04.426140 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:04.426140 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:25:04.426140 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:25:04.426140 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:25:04.426140 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:25:04.426140 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:25:04.426140 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:25:04.426140 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:25:04.426140 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:04.428819 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 240/489] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jun 15 19:25:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:25:04.430296 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74040279-9926-42bf-9be6-9311cca13603 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:04.430779 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74040279-9926-42bf-9be6-9311cca13603 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:04.431247 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74040279-9926-42bf-9be6-9311cca13603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:04.454524 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-74040279-9926-42bf-9be6-9311cca13603 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:04.456070 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-74040279-9926-42bf-9be6-9311cca13603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:04.546301 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 241/490] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:25:04.547714 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74040279-9926-42bf-9be6-9311cca13603 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:04.568216 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de0c47-3ae3-404a-abeb-4bb4dcae83d6 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:04.591888 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:04.593591 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:04.614415 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3884b5fc-dc25-4609-8aa9-25645511f864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:04.615684 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884b5fc-dc25-4609-8aa9-25645511f864 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:04.616152 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884b5fc-dc25-4609-8aa9-25645511f864 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:04.616754 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884b5fc-dc25-4609-8aa9-25645511f864 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:04.617965 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3884b5fc-dc25-4609-8aa9-25645511f864 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:04.618481 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3884b5fc-dc25-4609-8aa9-25645511f864 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:04.620281 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3884b5fc-dc25-4609-8aa9-25645511f864 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:04.632318 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 250/491] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:04.632318 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3884b5fc-dc25-4609-8aa9-25645511f864 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:04.671294 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74040279-9926-42bf-9be6-9311cca13603 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:04.706017 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2a975e-2bda-4e6f-972e-6bcad893873d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:04.706719 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a975e-2bda-4e6f-972e-6bcad893873d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:04.707001 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a975e-2bda-4e6f-972e-6bcad893873d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:04.707276 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a975e-2bda-4e6f-972e-6bcad893873d None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:04.708020 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2a975e-2bda-4e6f-972e-6bcad893873d None admin] RBAC: Adding query filter params (name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:04.708326 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2a975e-2bda-4e6f-972e-6bcad893873d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:04.709711 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2a975e-2bda-4e6f-972e-6bcad893873d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:04.717888 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 242/492] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jun 15 19:25:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:04.719138 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fecf3c8-1dc5-418c-8630-226c48353113 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:04.719422 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fecf3c8-1dc5-418c-8630-226c48353113 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:04.719694 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fecf3c8-1dc5-418c-8630-226c48353113 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:04.724830 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 243/493] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:04.725612 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fecf3c8-1dc5-418c-8630-226c48353113 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:04.746672 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3884b5fc-dc25-4609-8aa9-25645511f864 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:04.783323 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964266c-1ff6-4270-8eb8-109c026d90a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:04.784480 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964266c-1ff6-4270-8eb8-109c026d90a2 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:04.784927 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964266c-1ff6-4270-8eb8-109c026d90a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:04.785342 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964266c-1ff6-4270-8eb8-109c026d90a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:04.796743 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6964266c-1ff6-4270-8eb8-109c026d90a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:25:04.805019 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-6964266c-1ff6-4270-8eb8-109c026d90a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:25:04.805019 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:04.805019 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:04.805019 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:04.805019 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:04.805019 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:04.805019 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:04.805019 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:04.805019 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:04.805019 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:04.805019 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:04.805019 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:04.805019 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:25:04.805019 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:25:04.805019 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:25:04.805019 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:25:04.806112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:25:04.806112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:25:04.806112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:25:04.806112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:25:04.806112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:04.806112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:04.806112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:04.806112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:04.806112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:04.806112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:04.806112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:04.806112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:04.806112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:04.806112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:04.806112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:04.806112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:04.807100 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:04.807100 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:04.807100 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:04.807100 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:04.807100 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:25:04.807100 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:25:04.807100 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:25:04.807100 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:25:04.807100 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:25:04.807100 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:25:04.807100 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:25:04.807100 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:04.809193 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 251/494] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jun 15 19:25:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:04.810137 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964266c-1ff6-4270-8eb8-109c026d90a2 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:04.821098 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:04.836426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fecf3c8-1dc5-418c-8630-226c48353113 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:04.848997 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:04.875778 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432842c4-678e-44a1-b359-9d4998f115f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:04.877127 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432842c4-678e-44a1-b359-9d4998f115f4 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:04.877689 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432842c4-678e-44a1-b359-9d4998f115f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:04.878441 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432842c4-678e-44a1-b359-9d4998f115f4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:04.889452 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432842c4-678e-44a1-b359-9d4998f115f4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:04.891168 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432842c4-678e-44a1-b359-9d4998f115f4 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:04.912515 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-432842c4-678e-44a1-b359-9d4998f115f4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 19:25:04.912515 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:04.912515 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:04.912515 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:04.912515 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:04.912515 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:04.912515 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:04.912515 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:04.912515 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:04.912515 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:04.912515 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:04.912515 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:04.912515 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 19:25:04.912515 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 19:25:04.912515 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 19:25:04.912515 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 19:25:04.913645 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:04.913645 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:04.913645 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:25:04.913645 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:04.913645 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:25:04.913645 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:04.913645 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:04.913645 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:04.913645 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:04.913645 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:04.913645 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:04.913645 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:04.913645 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:04.913645 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:04.913645 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:04.913645 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:04.913645 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:04.913645 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:04.914795 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:04.914795 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:04.914795 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 19:25:04.914795 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 19:25:04.914795 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 19:25:04.914795 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 19:25:04.914795 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 19:25:04.914795 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 19:25:04.914795 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 19:25:04.914795 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:04.915563 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 244/495] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Jun 15 19:25:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:04.916578 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006be09-2a96-453d-a23a-b2b581023ca6 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:04.916896 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006be09-2a96-453d-a23a-b2b581023ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:04.917178 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006be09-2a96-453d-a23a-b2b581023ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:04.922144 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964266c-1ff6-4270-8eb8-109c026d90a2 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:04.934961 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-f006be09-2a96-453d-a23a-b2b581023ca6 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:04.936137 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f006be09-2a96-453d-a23a-b2b581023ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:04.957246 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e540970-697e-470d-aa51-90879ebfcbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:04.958313 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e540970-697e-470d-aa51-90879ebfcbbc None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:04.958769 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e540970-697e-470d-aa51-90879ebfcbbc None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:04.959238 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e540970-697e-470d-aa51-90879ebfcbbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:04.971681 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e540970-697e-470d-aa51-90879ebfcbbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:04.973352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e540970-697e-470d-aa51-90879ebfcbbc None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:04.991565 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-3e540970-697e-470d-aa51-90879ebfcbbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:04.991565 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:04.991565 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:04.991565 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:04.991565 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:04.991565 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:04.991565 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:04.991565 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:04.991565 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:04.991565 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:04.991565 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:04.991565 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:04.991565 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:04.991565 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:04.991565 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:04.992435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:04.992435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:04.992435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:04.992435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:04.992435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:04.992435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:04.992435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:04.992435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:04.992435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:04.992435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:04.992435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:04.992435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:04.992435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:04.992435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:04.992435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:04.992435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:04.992435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:04.993366 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:04.993366 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:04.993366 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:04.993366 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:04.995772 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 252/496] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:04.996763 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e540970-697e-470d-aa51-90879ebfcbbc None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:05.012238 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 245/497] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:25:05.013066 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f006be09-2a96-453d-a23a-b2b581023ca6 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:05.097506 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:05.105178 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:05.119127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e540970-697e-470d-aa51-90879ebfcbbc None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:05.122513 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f006be09-2a96-453d-a23a-b2b581023ca6 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:05.155874 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca738ae7-f296-4073-8c58-5444c20d6be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:05.156548 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca738ae7-f296-4073-8c58-5444c20d6be5 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:05.156834 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca738ae7-f296-4073-8c58-5444c20d6be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:05.156933 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95672c33-fcff-454d-a876-5498f5466b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:05.157116 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca738ae7-f296-4073-8c58-5444c20d6be5 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:05.157540 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95672c33-fcff-454d-a876-5498f5466b05 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:05.157825 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95672c33-fcff-454d-a876-5498f5466b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:05.157926 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca738ae7-f296-4073-8c58-5444c20d6be5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:05.158116 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95672c33-fcff-454d-a876-5498f5466b05 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:05.158220 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca738ae7-f296-4073-8c58-5444c20d6be5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:05.158950 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95672c33-fcff-454d-a876-5498f5466b05 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:05.159253 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95672c33-fcff-454d-a876-5498f5466b05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:05.159612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca738ae7-f296-4073-8c58-5444c20d6be5 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:05.160667 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95672c33-fcff-454d-a876-5498f5466b05 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:05.168033 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 253/498] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jun 15 19:25:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:05.173040 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca738ae7-f296-4073-8c58-5444c20d6be5 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:05.207975 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 246/499] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Jun 15 19:25:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:05.207975 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95672c33-fcff-454d-a876-5498f5466b05 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:05.283612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca738ae7-f296-4073-8c58-5444c20d6be5 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:05.307319 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95672c33-fcff-454d-a876-5498f5466b05 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:05.316788 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfed6f4c-898b-4d6c-a7fc-29f9bd11a55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:05.317451 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed6f4c-898b-4d6c-a7fc-29f9bd11a55e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:05.317742 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed6f4c-898b-4d6c-a7fc-29f9bd11a55e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:05.318019 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed6f4c-898b-4d6c-a7fc-29f9bd11a55e None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:05.318854 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfed6f4c-898b-4d6c-a7fc-29f9bd11a55e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:05.319171 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfed6f4c-898b-4d6c-a7fc-29f9bd11a55e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:05.320549 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfed6f4c-898b-4d6c-a7fc-29f9bd11a55e None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:05.334186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 254/500] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:05.335004 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfed6f4c-898b-4d6c-a7fc-29f9bd11a55e None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:05.346496 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9be62d-620b-4a15-a7a1-14c9b131c67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:05.347332 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9be62d-620b-4a15-a7a1-14c9b131c67e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:05.347663 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9be62d-620b-4a15-a7a1-14c9b131c67e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:05.347885 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9be62d-620b-4a15-a7a1-14c9b131c67e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:05.350608 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:05.351855 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:05.360094 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9be62d-620b-4a15-a7a1-14c9b131c67e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:25:05.367850 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-4e9be62d-620b-4a15-a7a1-14c9b131c67e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:25:05.367850 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:05.367850 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:05.367850 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:05.367850 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:05.367850 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:05.367850 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:05.367850 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:05.367850 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:05.367850 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:05.367850 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:05.367850 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:05.367850 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:25:05.367850 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:25:05.367850 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:25:05.367850 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:25:05.368917 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:25:05.368917 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:25:05.368917 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:25:05.368917 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:25:05.368917 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:05.368917 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:05.368917 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:05.368917 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:05.368917 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:05.368917 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:05.368917 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:05.368917 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:05.368917 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:05.368917 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:05.368917 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:05.368917 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:05.369780 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:05.369780 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:05.369780 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:05.369780 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:05.369780 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:25:05.369780 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:25:05.369780 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:25:05.369780 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:25:05.369780 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:25:05.369780 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:25:05.369780 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:25:05.369780 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:05.369780 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 247/501] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jun 15 19:25:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:05.369780 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9be62d-620b-4a15-a7a1-14c9b131c67e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:05.443116 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfed6f4c-898b-4d6c-a7fc-29f9bd11a55e None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:05.469442 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9be62d-620b-4a15-a7a1-14c9b131c67e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:05.478050 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c26edc-8143-49d2-b3f3-6afed198a943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:05.478929 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c26edc-8143-49d2-b3f3-6afed198a943 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:05.479210 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c26edc-8143-49d2-b3f3-6afed198a943 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:05.479524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c26edc-8143-49d2-b3f3-6afed198a943 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:05.489507 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c26edc-8143-49d2-b3f3-6afed198a943 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:05.490983 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c26edc-8143-49d2-b3f3-6afed198a943 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:05.501972 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe2ad65-b2ef-415c-9ae1-a950feff192b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:05.502639 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2ad65-b2ef-415c-9ae1-a950feff192b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:05.502951 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2ad65-b2ef-415c-9ae1-a950feff192b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:05.503228 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2ad65-b2ef-415c-9ae1-a950feff192b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:05.507818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-49c26edc-8143-49d2-b3f3-6afed198a943 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:05.507818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:05.507818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:05.507818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:05.507818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:05.507818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:05.507818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:05.507818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:05.507818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:05.507818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:05.507818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:05.507818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:05.507818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:05.507818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:05.507818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:05.508933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:05.508933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:05.508933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:05.508933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:05.508933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:05.508933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:05.508933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:05.508933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:05.508933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:05.508933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:05.508933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:05.508933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:05.508933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:05.508933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:05.508933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:05.508933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:05.508933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:05.509779 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:05.509779 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:05.509779 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:05.509779 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:05.509779 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 255/502] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:05.510484 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf16ca8-8e21-484b-b9f6-2d0ff774957f None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:05.511257 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf16ca8-8e21-484b-b9f6-2d0ff774957f None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:05.511527 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf16ca8-8e21-484b-b9f6-2d0ff774957f None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:05.512770 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 256/503] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:05.513456 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf16ca8-8e21-484b-b9f6-2d0ff774957f None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:05.569581 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:05.574535 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:05.577936 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe2ad65-b2ef-415c-9ae1-a950feff192b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:05.579441 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe2ad65-b2ef-415c-9ae1-a950feff192b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:05.613189 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf16ca8-8e21-484b-b9f6-2d0ff774957f None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:05.645673 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d199eae1-5369-4508-a348-56b877c3a13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:05.646473 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199eae1-5369-4508-a348-56b877c3a13d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:05.646757 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199eae1-5369-4508-a348-56b877c3a13d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:05.647022 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199eae1-5369-4508-a348-56b877c3a13d None admin] PATH_INFO: `/v3/users/glance` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:05.654356 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d199eae1-5369-4508-a348-56b877c3a13d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:05.655785 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d199eae1-5369-4508-a348-56b877c3a13d None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:05.671244 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-d199eae1-5369-4508-a348-56b877c3a13d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 15 19:25:05.671244 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:05.671244 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:05.671244 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:05.671244 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:05.671244 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:05.671244 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:05.671244 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:05.671244 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:05.671244 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:05.671244 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:05.671244 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:05.671244 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 19:25:05.671244 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 19:25:05.671244 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 19:25:05.671244 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 19:25:05.672324 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:05.672324 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:05.672324 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:25:05.672324 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:05.672324 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:25:05.672324 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:05.672324 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:05.672324 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:05.672324 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:05.672324 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:05.672324 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:05.672324 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:05.672324 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:05.672324 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:05.672324 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:05.672324 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:05.672324 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:05.672324 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:05.673434 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:05.673434 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:05.673434 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 19:25:05.673434 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 19:25:05.673434 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 19:25:05.673434 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 19:25:05.673434 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 19:25:05.673434 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 19:25:05.673434 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 15 19:25:05.673434 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:05.673434 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 257/504] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:25:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:05.673434 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d199eae1-5369-4508-a348-56b877c3a13d None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:05.684944 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 248/505] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:25:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 19:25:05.684944 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe2ad65-b2ef-415c-9ae1-a950feff192b None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:05.770734 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d199eae1-5369-4508-a348-56b877c3a13d None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:05.772584 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:05.779900 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe2ad65-b2ef-415c-9ae1-a950feff192b None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:05.793446 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:05.805748 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fdd83d-dd16-49b8-8613-5be4146849a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:05.806443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fdd83d-dd16-49b8-8613-5be4146849a4 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:05.806726 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fdd83d-dd16-49b8-8613-5be4146849a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:05.807005 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fdd83d-dd16-49b8-8613-5be4146849a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:05.808281 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fdd83d-dd16-49b8-8613-5be4146849a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:05.808590 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fdd83d-dd16-49b8-8613-5be4146849a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:05.809962 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fdd83d-dd16-49b8-8613-5be4146849a4 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:05.814038 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d6a47-5d0b-466b-86fb-d0e975357163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:05.814722 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d6a47-5d0b-466b-86fb-d0e975357163 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:05.815008 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d6a47-5d0b-466b-86fb-d0e975357163 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:05.815286 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d6a47-5d0b-466b-86fb-d0e975357163 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:05.816009 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22d6a47-5d0b-466b-86fb-d0e975357163 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:05.816317 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22d6a47-5d0b-466b-86fb-d0e975357163 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:05.817683 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22d6a47-5d0b-466b-86fb-d0e975357163 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:05.817941 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 258/506] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jun 15 19:25:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:05.819184 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca6096-f80b-4e3b-9115-c3c2c7790006 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:05.819471 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca6096-f80b-4e3b-9115-c3c2c7790006 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:05.819747 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca6096-f80b-4e3b-9115-c3c2c7790006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:05.832029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 249/507] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:05.832783 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d6a47-5d0b-466b-86fb-d0e975357163 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:05.841992 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-7eca6096-f80b-4e3b-9115-c3c2c7790006 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:05.843217 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-7eca6096-f80b-4e3b-9115-c3c2c7790006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:05.908786 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 259/508] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:25:05.909536 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca6096-f80b-4e3b-9115-c3c2c7790006 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:05.933028 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d6a47-5d0b-466b-86fb-d0e975357163 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:05.965905 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25677c71-a09e-4b94-9c8c-72342a4bb6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:05.966635 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25677c71-a09e-4b94-9c8c-72342a4bb6a8 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:05.966925 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25677c71-a09e-4b94-9c8c-72342a4bb6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:05.967207 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25677c71-a09e-4b94-9c8c-72342a4bb6a8 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:05.968057 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25677c71-a09e-4b94-9c8c-72342a4bb6a8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:05.968369 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25677c71-a09e-4b94-9c8c-72342a4bb6a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:05.969802 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25677c71-a09e-4b94-9c8c-72342a4bb6a8 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:05.987006 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 250/509] 10.222.0.15 () {56 vars in 1137 bytes} [Wed Jun 15 19:25:05 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:05.987797 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25677c71-a09e-4b94-9c8c-72342a4bb6a8 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:06.002755 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:06.017661 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca6096-f80b-4e3b-9115-c3c2c7790006 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:06.031806 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:06.051444 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dfa4ca-24ea-4814-96fe-fc1baceb52d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:06.052228 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dfa4ca-24ea-4814-96fe-fc1baceb52d0 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:06.052505 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dfa4ca-24ea-4814-96fe-fc1baceb52d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:06.052778 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dfa4ca-24ea-4814-96fe-fc1baceb52d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:06.063127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dfa4ca-24ea-4814-96fe-fc1baceb52d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:06.064819 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dfa4ca-24ea-4814-96fe-fc1baceb52d0 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:06.082346 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-31dfa4ca-24ea-4814-96fe-fc1baceb52d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:06.082346 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:06.082346 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:06.082346 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:06.082346 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:06.082346 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:06.082346 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:06.082346 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:06.082346 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:06.082346 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:06.082346 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:06.082346 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:06.082346 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:06.082346 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:06.082346 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:06.083199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:06.083199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:06.083199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:06.083199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:06.083199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:06.083199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:06.083199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:06.083199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:06.083199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:06.083199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:06.083199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:06.083199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:06.083199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:06.083199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:06.083199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:06.083199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:06.083199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:06.084045 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:06.084045 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:06.084045 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:06.084045 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:06.084045 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 260/510] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:06.084725 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dfa4ca-24ea-4814-96fe-fc1baceb52d0 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:06.089201 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25677c71-a09e-4b94-9c8c-72342a4bb6a8 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:06.122133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecec3de-281b-40fa-b4c7-d772a69b0431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:06.122923 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecec3de-281b-40fa-b4c7-d772a69b0431 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:06.123210 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecec3de-281b-40fa-b4c7-d772a69b0431 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:06.123490 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecec3de-281b-40fa-b4c7-d772a69b0431 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:06.133744 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecec3de-281b-40fa-b4c7-d772a69b0431 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:06.135242 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecec3de-281b-40fa-b4c7-d772a69b0431 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:06.153498 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-eecec3de-281b-40fa-b4c7-d772a69b0431 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:06.153498 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:06.153498 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:06.153498 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:06.153498 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:06.153498 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:06.153498 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:06.153498 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:06.153498 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:06.153498 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:06.153498 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:06.153498 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:06.153498 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:06.153498 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:06.153498 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:06.154518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:06.154518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:06.154518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:06.154518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:06.154518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:06.154518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:06.154518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:06.154518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:06.154518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:06.154518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:06.154518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:06.154518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:06.154518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:06.154518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:06.154518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:06.154518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:06.154518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:06.155363 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:06.155363 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:06.155363 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:06.155363 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:06.155363 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 251/511] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:06.155912 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecec3de-281b-40fa-b4c7-d772a69b0431 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:06.184949 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dfa4ca-24ea-4814-96fe-fc1baceb52d0 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:06.217851 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4618417f-941e-4a59-aecd-7002998f54a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:06.218676 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618417f-941e-4a59-aecd-7002998f54a6 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:06.218949 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618417f-941e-4a59-aecd-7002998f54a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:06.219228 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618417f-941e-4a59-aecd-7002998f54a6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:06.219603 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:06.226780 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4618417f-941e-4a59-aecd-7002998f54a6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:06.228235 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4618417f-941e-4a59-aecd-7002998f54a6 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:06.242577 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-4618417f-941e-4a59-aecd-7002998f54a6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 19:25:06.242577 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:06.242577 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:06.242577 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:06.242577 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:06.242577 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:06.242577 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:06.242577 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:06.242577 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:06.242577 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:06.242577 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:06.242577 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:06.242577 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 19:25:06.242577 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 19:25:06.242577 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 19:25:06.242577 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 19:25:06.243561 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:06.243561 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:06.243561 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:25:06.243561 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:06.243561 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:25:06.243561 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:06.243561 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:06.243561 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:06.243561 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:06.243561 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:06.243561 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:06.243561 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:06.243561 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:06.243561 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:06.243561 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:06.243561 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:06.243561 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:06.243561 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:06.244442 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:06.244442 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:06.244442 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 19:25:06.244442 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 19:25:06.244442 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 19:25:06.244442 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 19:25:06.244442 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 19:25:06.244442 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 19:25:06.244442 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 19:25:06.244442 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:06.244442 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 261/512] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:25:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:06.245537 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4618417f-941e-4a59-aecd-7002998f54a6 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:06.257639 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecec3de-281b-40fa-b4c7-d772a69b0431 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:06.273137 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:06.292646 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d38a77-3f57-4960-874c-5d07ef3d75a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:06.293437 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d38a77-3f57-4960-874c-5d07ef3d75a3 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:06.293724 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d38a77-3f57-4960-874c-5d07ef3d75a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:06.294005 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d38a77-3f57-4960-874c-5d07ef3d75a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:06.303826 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d38a77-3f57-4960-874c-5d07ef3d75a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:06.305288 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d38a77-3f57-4960-874c-5d07ef3d75a3 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:06.322447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-a6d38a77-3f57-4960-874c-5d07ef3d75a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:06.322447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:06.322447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:06.322447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:06.322447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:06.322447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:06.322447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:06.322447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:06.322447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:06.322447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:06.322447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:06.322447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:06.322447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:06.322447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:06.322447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:06.324045 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:06.324045 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:06.324045 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:06.324045 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:06.324045 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:06.324045 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:06.324045 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:06.324045 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:06.324045 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:06.324045 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:06.324045 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:06.324045 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:06.324045 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:06.324045 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:06.324045 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:06.324045 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:06.324045 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:06.325974 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:06.325974 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:06.325974 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:06.325974 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:06.325974 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 252/513] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:06.325974 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d38a77-3f57-4960-874c-5d07ef3d75a3 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:06.351524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4618417f-941e-4a59-aecd-7002998f54a6 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:06.385513 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1789e4-8848-4166-af48-a058b91c4295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:06.386664 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1789e4-8848-4166-af48-a058b91c4295 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:06.386944 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1789e4-8848-4166-af48-a058b91c4295 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:06.387213 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1789e4-8848-4166-af48-a058b91c4295 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:06.425005 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d38a77-3f57-4960-874c-5d07ef3d75a3 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:06.443282 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:06.454779 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1789e4-8848-4166-af48-a058b91c4295 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:06.456241 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1789e4-8848-4166-af48-a058b91c4295 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:06.461064 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d884f3-2f4d-4eef-aa19-aadc69eede34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:06.461722 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d884f3-2f4d-4eef-aa19-aadc69eede34 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:06.462004 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d884f3-2f4d-4eef-aa19-aadc69eede34 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:06.462303 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d884f3-2f4d-4eef-aa19-aadc69eede34 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:06.463014 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d884f3-2f4d-4eef-aa19-aadc69eede34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:06.463318 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d884f3-2f4d-4eef-aa19-aadc69eede34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:06.465260 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d884f3-2f4d-4eef-aa19-aadc69eede34 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:06.476831 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 253/514] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:06.477515 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d884f3-2f4d-4eef-aa19-aadc69eede34 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:06.487253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:06.559894 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 262/515] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:25:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 19:25:06.560925 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1789e4-8848-4166-af48-a058b91c4295 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:06.576143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d884f3-2f4d-4eef-aa19-aadc69eede34 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:06.609597 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd2410c-88b5-42db-aace-f920fb44f1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:06.610293 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2410c-88b5-42db-aace-f920fb44f1ab None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:06.610581 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2410c-88b5-42db-aace-f920fb44f1ab None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:06.610864 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2410c-88b5-42db-aace-f920fb44f1ab None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:06.611588 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd2410c-88b5-42db-aace-f920fb44f1ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:06.611899 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd2410c-88b5-42db-aace-f920fb44f1ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:06.613271 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd2410c-88b5-42db-aace-f920fb44f1ab None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:06.624649 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 254/516] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:06.625428 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd2410c-88b5-42db-aace-f920fb44f1ab None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:06.661862 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1789e4-8848-4166-af48-a058b91c4295 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:06.663518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:06.687545 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:06.696251 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d965b0f7-ae31-46c6-acd4-08b3d58466dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:06.696915 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965b0f7-ae31-46c6-acd4-08b3d58466dc None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:06.697202 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965b0f7-ae31-46c6-acd4-08b3d58466dc None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:06.697477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965b0f7-ae31-46c6-acd4-08b3d58466dc None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:06.698385 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d965b0f7-ae31-46c6-acd4-08b3d58466dc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:06.698721 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d965b0f7-ae31-46c6-acd4-08b3d58466dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:06.700142 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d965b0f7-ae31-46c6-acd4-08b3d58466dc None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:06.723049 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd2410c-88b5-42db-aace-f920fb44f1ab None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:06.736655 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 263/517] 10.222.0.15 () {56 vars in 1137 bytes} [Wed Jun 15 19:25:06 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:06.737576 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d965b0f7-ae31-46c6-acd4-08b3d58466dc None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:06.756302 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa5b02-286e-496a-ae9b-51aaeefd72b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:06.756960 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa5b02-286e-496a-ae9b-51aaeefd72b7 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:06.757241 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa5b02-286e-496a-ae9b-51aaeefd72b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:06.757513 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa5b02-286e-496a-ae9b-51aaeefd72b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:06.758253 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa5b02-286e-496a-ae9b-51aaeefd72b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:06.758824 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa5b02-286e-496a-ae9b-51aaeefd72b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:06.760187 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa5b02-286e-496a-ae9b-51aaeefd72b7 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:06.771127 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 255/518] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:06.771911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa5b02-286e-496a-ae9b-51aaeefd72b7 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:06.841744 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d965b0f7-ae31-46c6-acd4-08b3d58466dc None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:06.871175 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:06.874788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa5b02-286e-496a-ae9b-51aaeefd72b7 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:06.876481 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1b6b4f-e52a-4626-822d-f78c71a41b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:06.877269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b6b4f-e52a-4626-822d-f78c71a41b58 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:06.877562 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b6b4f-e52a-4626-822d-f78c71a41b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:06.877845 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b6b4f-e52a-4626-822d-f78c71a41b58 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:06.884829 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1b6b4f-e52a-4626-822d-f78c71a41b58 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:06.886837 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1b6b4f-e52a-4626-822d-f78c71a41b58 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:06.901270 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-5d1b6b4f-e52a-4626-822d-f78c71a41b58 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:06.901270 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:06.901270 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:06.901270 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:06.901270 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:06.901270 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:06.901270 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:06.901270 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:06.901270 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:06.901270 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:06.901270 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:06.901270 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:06.901270 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 19:25:06.901270 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 19:25:06.901270 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 19:25:06.902648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 19:25:06.902648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:06.902648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:06.902648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:06.902648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:06.902648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:06.902648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:06.902648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:06.902648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:06.902648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:06.902648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:06.902648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:06.902648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:06.902648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:06.902648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:06.902648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:06.902648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:06.903489 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 19:25:06.903489 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 19:25:06.903489 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:25:06.903489 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:25:06.903489 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:25:06.903489 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:25:06.903489 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:06.903489 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:06.903489 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 264/519] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:25:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:06.903945 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1b6b4f-e52a-4626-822d-f78c71a41b58 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:06.913019 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29adce7-7094-4289-8e78-c1cb84f5b1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:06.913809 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29adce7-7094-4289-8e78-c1cb84f5b1d1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:06.914108 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29adce7-7094-4289-8e78-c1cb84f5b1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:06.914396 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29adce7-7094-4289-8e78-c1cb84f5b1d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:06.924765 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29adce7-7094-4289-8e78-c1cb84f5b1d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:06.926271 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29adce7-7094-4289-8e78-c1cb84f5b1d1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:06.943363 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-d29adce7-7094-4289-8e78-c1cb84f5b1d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:06.943363 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:06.943363 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:06.943363 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:06.943363 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:06.943363 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:06.943363 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:06.943363 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:06.943363 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:06.943363 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:06.943363 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:06.943363 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:06.943363 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:06.943363 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:06.943363 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:06.944334 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:06.944334 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:06.944334 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:06.944334 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:06.944334 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:06.944334 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:06.944334 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:06.944334 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:06.944334 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:06.944334 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:06.944334 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:06.944334 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:06.944334 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:06.944334 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:06.944334 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:06.944334 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:06.944334 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:06.945206 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:06.945206 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:06.945206 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:06.945206 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:06.945206 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 256/520] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:06.946680 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29adce7-7094-4289-8e78-c1cb84f5b1d1 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:06.950471 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:07.003695 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1b6b4f-e52a-4626-822d-f78c71a41b58 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:07.037998 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fc20c7-faa5-4b66-b3f1-a487f9107d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:07.038807 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc20c7-faa5-4b66-b3f1-a487f9107d51 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:07.039088 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc20c7-faa5-4b66-b3f1-a487f9107d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:07.039367 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc20c7-faa5-4b66-b3f1-a487f9107d51 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:07.044917 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29adce7-7094-4289-8e78-c1cb84f5b1d1 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:07.046696 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fc20c7-faa5-4b66-b3f1-a487f9107d51 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:07.048149 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fc20c7-faa5-4b66-b3f1-a487f9107d51 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:07.062012 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-b7fc20c7-faa5-4b66-b3f1-a487f9107d51 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 19:25:07.062012 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:07.062012 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:07.062012 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:07.062012 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:07.062012 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:07.062012 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:07.062012 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:07.062012 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:07.062012 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:07.062012 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:07.062012 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:07.062012 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 19:25:07.062012 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 19:25:07.062012 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 19:25:07.062012 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 19:25:07.062922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:07.062922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:07.062922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:25:07.062922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:07.062922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:25:07.062922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:07.062922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:07.062922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:07.062922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:07.062922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:07.062922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:07.062922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:07.062922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:07.062922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:07.062922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:07.062922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:07.062922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:07.062922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:07.063820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:07.063820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:07.063820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 19:25:07.063820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 19:25:07.063820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 19:25:07.063820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 19:25:07.063820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 19:25:07.063820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 19:25:07.063820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 19:25:07.063820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:07.063820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 265/521] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jun 15 19:25:06 2022] GET /identity/v3/users/neutron => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:07.065075 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fc20c7-faa5-4b66-b3f1-a487f9107d51 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:07.083425 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b702d-bf6b-49f0-a804-1c35e23e14ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:07.084075 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b702d-bf6b-49f0-a804-1c35e23e14ae None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:07.084354 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b702d-bf6b-49f0-a804-1c35e23e14ae None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:07.084625 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b702d-bf6b-49f0-a804-1c35e23e14ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:07.103486 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:07.156344 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b702d-bf6b-49f0-a804-1c35e23e14ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:07.157880 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b702d-bf6b-49f0-a804-1c35e23e14ae None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:07.168458 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fc20c7-faa5-4b66-b3f1-a487f9107d51 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:07.171519 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:07.202434 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b318e735-138b-4fae-9f7d-69927dc5b390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:07.203038 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318e735-138b-4fae-9f7d-69927dc5b390 None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:07.203317 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318e735-138b-4fae-9f7d-69927dc5b390 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:07.203588 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318e735-138b-4fae-9f7d-69927dc5b390 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:07.204287 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b318e735-138b-4fae-9f7d-69927dc5b390 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:07.205710 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b318e735-138b-4fae-9f7d-69927dc5b390 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:07.225838 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 266/522] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Jun 15 19:25:07 2022] POST /identity/v3/users => generated 247 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:25:07.226666 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b318e735-138b-4fae-9f7d-69927dc5b390 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:07.266180 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 257/523] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:25:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 19:25:07.266180 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b702d-bf6b-49f0-a804-1c35e23e14ae None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:07.321390 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:07.348890 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b318e735-138b-4fae-9f7d-69927dc5b390 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:07.386434 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66231f0e-e356-4fe0-8335-70860ce5f053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:07.387126 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66231f0e-e356-4fe0-8335-70860ce5f053 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:07.387408 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66231f0e-e356-4fe0-8335-70860ce5f053 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:07.387685 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66231f0e-e356-4fe0-8335-70860ce5f053 None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:07.388480 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66231f0e-e356-4fe0-8335-70860ce5f053 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:07.388792 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66231f0e-e356-4fe0-8335-70860ce5f053 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:07.388886 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b702d-bf6b-49f0-a804-1c35e23e14ae None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:07.390266 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66231f0e-e356-4fe0-8335-70860ce5f053 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:07.408276 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 267/524] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jun 15 19:25:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:07.408916 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66231f0e-e356-4fe0-8335-70860ce5f053 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:07.429704 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c91cdc6-1891-45f5-bda0-2cd0f4459909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:07.430407 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91cdc6-1891-45f5-bda0-2cd0f4459909 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:07.430703 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91cdc6-1891-45f5-bda0-2cd0f4459909 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:07.431005 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91cdc6-1891-45f5-bda0-2cd0f4459909 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:07.431423 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:07.431749 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c91cdc6-1891-45f5-bda0-2cd0f4459909 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:07.432274 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c91cdc6-1891-45f5-bda0-2cd0f4459909 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:07.434026 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c91cdc6-1891-45f5-bda0-2cd0f4459909 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:07.450173 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 258/525] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:07.451069 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c91cdc6-1891-45f5-bda0-2cd0f4459909 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:07.529330 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66231f0e-e356-4fe0-8335-70860ce5f053 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:07.565275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34ac4e3-ce5b-4b99-898c-ca145625b61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:07.565941 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34ac4e3-ce5b-4b99-898c-ca145625b61c None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:07.566288 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34ac4e3-ce5b-4b99-898c-ca145625b61c None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:07.566571 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34ac4e3-ce5b-4b99-898c-ca145625b61c None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:07.566768 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c91cdc6-1891-45f5-bda0-2cd0f4459909 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:07.567406 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34ac4e3-ce5b-4b99-898c-ca145625b61c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:07.567709 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34ac4e3-ce5b-4b99-898c-ca145625b61c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:07.569147 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34ac4e3-ce5b-4b99-898c-ca145625b61c None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:07.577808 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:07.587309 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 268/526] 10.222.0.15 () {56 vars in 1139 bytes} [Wed Jun 15 19:25:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:07.588136 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34ac4e3-ce5b-4b99-898c-ca145625b61c None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:07.609108 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a490c-1887-4d07-bb73-b163524b1a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:07.609908 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a490c-1887-4d07-bb73-b163524b1a40 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:07.610228 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a490c-1887-4d07-bb73-b163524b1a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:07.610508 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a490c-1887-4d07-bb73-b163524b1a40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:07.621308 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341a490c-1887-4d07-bb73-b163524b1a40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:07.622906 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341a490c-1887-4d07-bb73-b163524b1a40 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:07.640543 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-341a490c-1887-4d07-bb73-b163524b1a40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:07.640543 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:07.640543 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:07.640543 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:07.640543 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:07.640543 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:07.640543 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:07.640543 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:07.640543 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:07.640543 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:07.640543 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:07.640543 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:07.640543 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:07.640543 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:07.640543 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:07.641438 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:07.641438 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:07.641438 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:07.641438 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:07.641438 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:07.641438 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:07.641438 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:07.641438 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:07.641438 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:07.641438 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:07.641438 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:07.641438 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:07.641438 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:07.641438 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:07.641438 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:07.641438 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:07.641438 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:07.642815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:07.642815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:07.642815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:07.642815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:07.642815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 259/527] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:07.642815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a490c-1887-4d07-bb73-b163524b1a40 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:07.678765 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:07.703235 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34ac4e3-ce5b-4b99-898c-ca145625b61c None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:07.739692 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029339c7-59d1-423c-87ed-ffc96d74ac15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:07.740363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029339c7-59d1-423c-87ed-ffc96d74ac15 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:07.740726 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029339c7-59d1-423c-87ed-ffc96d74ac15 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:07.741002 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029339c7-59d1-423c-87ed-ffc96d74ac15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:07.741941 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029339c7-59d1-423c-87ed-ffc96d74ac15 None admin] RBAC: Adding query filter params (role.id=a8df9714563b435e9fce55a04e74ae5e, scope.project.id=7fb8cc0eb560465e972ffd45867ee73b, user.id=3e74ac44e5814122a3226ac023f3688a) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:07.742279 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029339c7-59d1-423c-87ed-ffc96d74ac15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:07.743804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029339c7-59d1-423c-87ed-ffc96d74ac15 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:07.747392 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a490c-1887-4d07-bb73-b163524b1a40 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:07.749867 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 269/528] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:25:07 2022] GET /identity/v3/role_assignments?user.id=3e74ac44e5814122a3226ac023f3688a&scope.project.id=7fb8cc0eb560465e972ffd45867ee73b&role.id=a8df9714563b435e9fce55a04e74ae5e => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:07.750729 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029339c7-59d1-423c-87ed-ffc96d74ac15 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:07.813970 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb8b77-7b0c-435c-8689-6a1b69208a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:07.815021 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb8b77-7b0c-435c-8689-6a1b69208a01 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:07.815771 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb8b77-7b0c-435c-8689-6a1b69208a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:07.816063 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb8b77-7b0c-435c-8689-6a1b69208a01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:07.827251 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdb8b77-7b0c-435c-8689-6a1b69208a01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:07.828751 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdb8b77-7b0c-435c-8689-6a1b69208a01 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:07.853980 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-1bdb8b77-7b0c-435c-8689-6a1b69208a01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:07.853980 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:07.853980 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:07.853980 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:07.853980 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:07.853980 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:07.853980 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:07.853980 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:07.853980 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:07.853980 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:07.853980 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:07.853980 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:07.853980 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 19:25:07.853980 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 19:25:07.853980 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 19:25:07.854989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 19:25:07.854989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:07.854989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:07.854989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:07.854989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:07.854989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:07.854989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:07.854989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:07.854989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:07.854989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:07.854989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:07.854989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:07.854989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:07.854989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:07.854989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:07.854989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:07.854989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:07.855847 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 19:25:07.855847 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 19:25:07.855847 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:25:07.855847 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:25:07.855847 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:25:07.855847 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:25:07.855847 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:07.855847 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:07.858989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 260/529] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:25:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:07.859912 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb8b77-7b0c-435c-8689-6a1b69208a01 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:07.884076 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029339c7-59d1-423c-87ed-ffc96d74ac15 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:07.890074 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:07.929893 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda6006a-e373-40b0-8c91-4769438ed7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:07.931220 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda6006a-e373-40b0-8c91-4769438ed7ee None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:07.931714 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda6006a-e373-40b0-8c91-4769438ed7ee None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:07.932322 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda6006a-e373-40b0-8c91-4769438ed7ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:07.946856 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:07.973676 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb8b77-7b0c-435c-8689-6a1b69208a01 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:08.018130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91f86a-91d4-44c7-bef0-ddd22d0c90d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:08.019733 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91f86a-91d4-44c7-bef0-ddd22d0c90d5 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:08.020465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91f86a-91d4-44c7-bef0-ddd22d0c90d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:08.020924 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91f86a-91d4-44c7-bef0-ddd22d0c90d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:08.021058 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda6006a-e373-40b0-8c91-4769438ed7ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:08.021965 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef91f86a-91d4-44c7-bef0-ddd22d0c90d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:08.022640 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef91f86a-91d4-44c7-bef0-ddd22d0c90d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:08.022744 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda6006a-e373-40b0-8c91-4769438ed7ee None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:08.024295 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef91f86a-91d4-44c7-bef0-ddd22d0c90d5 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:08.048731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 261/530] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:08.048731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91f86a-91d4-44c7-bef0-ddd22d0c90d5 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:08.142364 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:08.162176 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 270/531] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:25:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 19:25:08.162176 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda6006a-e373-40b0-8c91-4769438ed7ee None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:08.172551 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91f86a-91d4-44c7-bef0-ddd22d0c90d5 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:08.214827 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc614e-6777-40c5-863f-e73c8843d5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:08.215776 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc614e-6777-40c5-863f-e73c8843d5cf None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:08.216206 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc614e-6777-40c5-863f-e73c8843d5cf None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:08.216640 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc614e-6777-40c5-863f-e73c8843d5cf None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:08.217592 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cc614e-6777-40c5-863f-e73c8843d5cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:08.218069 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cc614e-6777-40c5-863f-e73c8843d5cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:08.219688 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cc614e-6777-40c5-863f-e73c8843d5cf None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:08.223496 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:08.243983 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 262/532] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jun 15 19:25:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:08.245190 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc614e-6777-40c5-863f-e73c8843d5cf None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:08.301419 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda6006a-e373-40b0-8c91-4769438ed7ee None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:08.350172 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011f7e19-42e4-4f36-a230-8b846947f74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:08.351901 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011f7e19-42e4-4f36-a230-8b846947f74e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:08.352188 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011f7e19-42e4-4f36-a230-8b846947f74e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:08.352464 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011f7e19-42e4-4f36-a230-8b846947f74e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:08.363988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc614e-6777-40c5-863f-e73c8843d5cf None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:08.365568 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011f7e19-42e4-4f36-a230-8b846947f74e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:08.368204 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011f7e19-42e4-4f36-a230-8b846947f74e None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:08.387747 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-011f7e19-42e4-4f36-a230-8b846947f74e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:08.387747 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:08.387747 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:08.387747 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:08.387747 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:08.387747 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:08.387747 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:08.387747 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:08.387747 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:08.387747 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:08.387747 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:08.387747 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:08.387747 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 19:25:08.387747 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 19:25:08.387747 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 19:25:08.388686 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 19:25:08.388686 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:08.388686 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:08.388686 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:08.388686 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:08.388686 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:08.388686 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:08.388686 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:08.388686 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:08.388686 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:08.388686 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:08.388686 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:08.388686 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:08.388686 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:08.388686 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:08.388686 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:08.388686 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:08.389535 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 19:25:08.389535 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 19:25:08.389535 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:25:08.389535 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:25:08.389535 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:25:08.389535 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:25:08.389535 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:08.389535 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:08.392857 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 271/533] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:25:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:08.394072 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011f7e19-42e4-4f36-a230-8b846947f74e None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:08.405803 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d23a6-3386-45fc-8fde-e20868cec37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:08.406913 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d23a6-3386-45fc-8fde-e20868cec37d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:08.408067 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d23a6-3386-45fc-8fde-e20868cec37d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:08.408509 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d23a6-3386-45fc-8fde-e20868cec37d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:08.424787 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d23a6-3386-45fc-8fde-e20868cec37d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:08.426926 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d23a6-3386-45fc-8fde-e20868cec37d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:08.430853 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:08.455690 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-5c9d23a6-3386-45fc-8fde-e20868cec37d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:08.455690 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:08.455690 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:08.455690 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:08.455690 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:08.455690 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:08.455690 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:08.455690 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:08.455690 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:08.455690 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:08.455690 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:08.455690 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:08.455690 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:08.455690 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:08.455690 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:08.456731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:08.456731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:08.456731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:08.456731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:08.456731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:08.456731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:08.456731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:08.456731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:08.456731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:08.456731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:08.456731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:08.456731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:08.456731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:08.456731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:08.456731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:08.456731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:08.456731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:08.457565 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:08.457565 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:08.457565 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:08.457565 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:08.460398 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 263/534] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:08.461381 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d23a6-3386-45fc-8fde-e20868cec37d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:08.517733 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011f7e19-42e4-4f36-a230-8b846947f74e None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:08.519421 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:08.555035 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56da7c1-9561-4e9e-9c06-0668b561cdd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:08.555789 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56da7c1-9561-4e9e-9c06-0668b561cdd9 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:08.556150 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56da7c1-9561-4e9e-9c06-0668b561cdd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:08.556500 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56da7c1-9561-4e9e-9c06-0668b561cdd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:08.557959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56da7c1-9561-4e9e-9c06-0668b561cdd9 None admin] RBAC: Adding query filter params (role.id=a8df9714563b435e9fce55a04e74ae5e, scope.project.id=7fb8cc0eb560465e972ffd45867ee73b, user.id=8bfca4d302aa43249609513f35d55d7a) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:08.558381 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56da7c1-9561-4e9e-9c06-0668b561cdd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:08.559892 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56da7c1-9561-4e9e-9c06-0668b561cdd9 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:08.566083 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 272/535] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:25:08 2022] GET /identity/v3/role_assignments?user.id=8bfca4d302aa43249609513f35d55d7a&scope.project.id=7fb8cc0eb560465e972ffd45867ee73b&role.id=a8df9714563b435e9fce55a04e74ae5e => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:08.567028 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56da7c1-9561-4e9e-9c06-0668b561cdd9 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:08.574638 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d23a6-3386-45fc-8fde-e20868cec37d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:08.615771 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a90a9e-1e2f-4fff-8c00-ee28784058a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:08.616814 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a90a9e-1e2f-4fff-8c00-ee28784058a1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:08.617289 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a90a9e-1e2f-4fff-8c00-ee28784058a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:08.617800 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a90a9e-1e2f-4fff-8c00-ee28784058a1 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:08.618830 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a90a9e-1e2f-4fff-8c00-ee28784058a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:08.619311 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a90a9e-1e2f-4fff-8c00-ee28784058a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:08.621742 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a90a9e-1e2f-4fff-8c00-ee28784058a1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:08.636078 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 264/536] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jun 15 19:25:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:08.644142 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a90a9e-1e2f-4fff-8c00-ee28784058a1 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:08.680887 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56da7c1-9561-4e9e-9c06-0668b561cdd9 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:08.722510 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cfd7bf-ebee-478b-b29f-1172f2bbe2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:08.722510 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfd7bf-ebee-478b-b29f-1172f2bbe2ae None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:08.722510 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfd7bf-ebee-478b-b29f-1172f2bbe2ae None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:08.722510 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfd7bf-ebee-478b-b29f-1172f2bbe2ae None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:08.722943 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cfd7bf-ebee-478b-b29f-1172f2bbe2ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:08.722943 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cfd7bf-ebee-478b-b29f-1172f2bbe2ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:08.722943 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cfd7bf-ebee-478b-b29f-1172f2bbe2ae None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:08.738126 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 273/537] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:08.740626 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cfd7bf-ebee-478b-b29f-1172f2bbe2ae None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:08.749101 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:08.778878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:08.853509 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cfd7bf-ebee-478b-b29f-1172f2bbe2ae None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:08.878416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a90a9e-1e2f-4fff-8c00-ee28784058a1 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:08.891499 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10f720-081e-4c1e-a3e7-00a84d422622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:08.892359 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10f720-081e-4c1e-a3e7-00a84d422622 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:08.892645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10f720-081e-4c1e-a3e7-00a84d422622 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:08.892922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10f720-081e-4c1e-a3e7-00a84d422622 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:08.900214 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b10f720-081e-4c1e-a3e7-00a84d422622 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:08.902212 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b10f720-081e-4c1e-a3e7-00a84d422622 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:08.916645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-6b10f720-081e-4c1e-a3e7-00a84d422622 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:08.916645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:08.916645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:08.916645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:08.916645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:08.916645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:08.916645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:08.916645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:08.916645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:08.916645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:08.916645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:08.916645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:08.916645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 19:25:08.916645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 19:25:08.916645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 19:25:08.917603 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 19:25:08.917603 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:08.917603 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:08.917603 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:08.917603 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:08.917603 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:08.917603 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:08.917603 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:08.917603 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:08.917603 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:08.917603 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:08.917603 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:08.917603 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:08.917603 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:08.917603 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:08.917603 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:08.917603 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:08.918458 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 19:25:08.918458 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 19:25:08.918458 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:25:08.918458 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:25:08.918458 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:25:08.918458 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:25:08.918458 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:08.918458 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:08.918458 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 274/538] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:25:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:08.923995 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10f720-081e-4c1e-a3e7-00a84d422622 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:08.938813 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dce401-42bd-45c3-9c49-7c0b4c49a2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:08.939586 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dce401-42bd-45c3-9c49-7c0b4c49a2c1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:08.939946 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dce401-42bd-45c3-9c49-7c0b4c49a2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:08.940312 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dce401-42bd-45c3-9c49-7c0b4c49a2c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:08.941231 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dce401-42bd-45c3-9c49-7c0b4c49a2c1 None admin] RBAC: Adding query filter params (role.id=a8df9714563b435e9fce55a04e74ae5e, scope.project.id=7fb8cc0eb560465e972ffd45867ee73b, user.id=ec5882def40a4e0d8a792eecc12440a5) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:08.941611 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dce401-42bd-45c3-9c49-7c0b4c49a2c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:08.943219 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dce401-42bd-45c3-9c49-7c0b4c49a2c1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:08.956444 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 265/539] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:25:08 2022] GET /identity/v3/role_assignments?user.id=ec5882def40a4e0d8a792eecc12440a5&scope.project.id=7fb8cc0eb560465e972ffd45867ee73b&role.id=a8df9714563b435e9fce55a04e74ae5e => generated 259 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:09.067310 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:09.090821 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10f720-081e-4c1e-a3e7-00a84d422622 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:09.138010 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b884c96-8a7d-4973-a0d8-d681bb36fb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:09.139056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b884c96-8a7d-4973-a0d8-d681bb36fb6a None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:09.139515 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b884c96-8a7d-4973-a0d8-d681bb36fb6a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:09.139965 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b884c96-8a7d-4973-a0d8-d681bb36fb6a None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:09.140935 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b884c96-8a7d-4973-a0d8-d681bb36fb6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:09.141416 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b884c96-8a7d-4973-a0d8-d681bb36fb6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:09.143106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b884c96-8a7d-4973-a0d8-d681bb36fb6a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:09.162235 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 275/540] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jun 15 19:25:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:25:09.168230 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dce401-42bd-45c3-9c49-7c0b4c49a2c1 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:09.309652 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dce401-42bd-45c3-9c49-7c0b4c49a2c1 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:09.354068 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf07b8f-53c8-4152-bc1e-0ed061f83411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:09.358271 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:09.359079 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf07b8f-53c8-4152-bc1e-0ed061f83411 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:09.359348 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf07b8f-53c8-4152-bc1e-0ed061f83411 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:09.362467 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf07b8f-53c8-4152-bc1e-0ed061f83411 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:09.363334 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf07b8f-53c8-4152-bc1e-0ed061f83411 None admin] RBAC: Adding query filter params (role.id=a8df9714563b435e9fce55a04e74ae5e, scope.project.id=7fb8cc0eb560465e972ffd45867ee73b, user.id=5c63e0b6ceed4f7cb2e6f9269eb85795) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:09.363655 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf07b8f-53c8-4152-bc1e-0ed061f83411 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:09.365112 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf07b8f-53c8-4152-bc1e-0ed061f83411 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:09.371936 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 266/541] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:25:09 2022] GET /identity/v3/role_assignments?user.id=5c63e0b6ceed4f7cb2e6f9269eb85795&scope.project.id=7fb8cc0eb560465e972ffd45867ee73b&role.id=a8df9714563b435e9fce55a04e74ae5e => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:09.526570 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed45da7d-e114-4ddc-97c9-6ffeff53d182 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:09.527175 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed45da7d-e114-4ddc-97c9-6ffeff53d182 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:09.527596 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed45da7d-e114-4ddc-97c9-6ffeff53d182 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:09.529458 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 276/542] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:09.536117 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b680a38-b818-471b-8173-bfbaea51df77 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:09.536492 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b680a38-b818-471b-8173-bfbaea51df77 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:09.536838 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b680a38-b818-471b-8173-bfbaea51df77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:09.566572 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-1b680a38-b818-471b-8173-bfbaea51df77 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:09.567832 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1b680a38-b818-471b-8173-bfbaea51df77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:09.649008 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 267/543] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:25:09.695363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed45da7d-e114-4ddc-97c9-6ffeff53d182 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:09.832268 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed45da7d-e114-4ddc-97c9-6ffeff53d182 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:09.880431 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ce3b0-8916-4db8-a8b5-96d567c0e721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:09.881462 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ce3b0-8916-4db8-a8b5-96d567c0e721 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:09.881884 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ce3b0-8916-4db8-a8b5-96d567c0e721 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:09.882326 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ce3b0-8916-4db8-a8b5-96d567c0e721 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:09.902985 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765ce3b0-8916-4db8-a8b5-96d567c0e721 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:25:09.910918 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-765ce3b0-8916-4db8-a8b5-96d567c0e721 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:25:09.910918 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:09.910918 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:09.910918 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:09.910918 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:09.910918 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:09.910918 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:09.910918 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:09.910918 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:09.910918 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:09.910918 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:09.910918 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:09.910918 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:25:09.910918 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:25:09.910918 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:25:09.910918 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:25:09.911812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:25:09.911812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:25:09.911812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:25:09.911812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:25:09.911812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:09.911812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:09.911812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:09.911812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:09.911812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:09.911812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:09.911812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:09.911812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:09.911812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:09.911812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:09.911812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:09.911812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:09.912605 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:09.912605 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:09.912605 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:09.912605 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:09.912605 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:25:09.912605 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:25:09.912605 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:25:09.912605 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:25:09.912605 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:25:09.912605 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:25:09.912605 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:25:09.912605 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:09.915098 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 277/544] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jun 15 19:25:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:09.921543 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b680a38-b818-471b-8173-bfbaea51df77 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:10.072009 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b680a38-b818-471b-8173-bfbaea51df77 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:10.086161 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:10.115684 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14363570-5361-4153-995e-8864ecd17cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:10.116382 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14363570-5361-4153-995e-8864ecd17cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:10.116668 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14363570-5361-4153-995e-8864ecd17cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:10.116946 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14363570-5361-4153-995e-8864ecd17cc8 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:10.117702 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14363570-5361-4153-995e-8864ecd17cc8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:10.118010 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14363570-5361-4153-995e-8864ecd17cc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:10.119435 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14363570-5361-4153-995e-8864ecd17cc8 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:10.134034 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 268/545] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jun 15 19:25:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:10.139261 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ce3b0-8916-4db8-a8b5-96d567c0e721 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:10.149984 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:10.290006 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ce3b0-8916-4db8-a8b5-96d567c0e721 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:10.339451 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc77f51-7cb9-4e67-aaff-59c2250797f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:10.340192 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc77f51-7cb9-4e67-aaff-59c2250797f6 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:10.340481 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc77f51-7cb9-4e67-aaff-59c2250797f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:10.341232 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc77f51-7cb9-4e67-aaff-59c2250797f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:10.362286 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc77f51-7cb9-4e67-aaff-59c2250797f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:10.363778 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc77f51-7cb9-4e67-aaff-59c2250797f6 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:10.393417 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-2cc77f51-7cb9-4e67-aaff-59c2250797f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:10.393417 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:10.393417 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:10.393417 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:10.393417 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:10.393417 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:10.393417 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:10.393417 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:10.393417 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:10.393417 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:10.393417 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:10.393417 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:10.393417 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:10.393417 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:10.393417 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:10.394370 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:10.394370 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:10.394370 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:10.394370 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:10.394370 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:10.394370 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:10.394370 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:10.394370 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:10.394370 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:10.394370 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:10.394370 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:10.394370 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:10.394370 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:10.394370 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:10.394370 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:10.394370 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:10.394370 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:10.395198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:10.395198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:10.395198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:10.395198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:10.395198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 278/546] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:10.400340 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14363570-5361-4153-995e-8864ecd17cc8 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:10.520103 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14363570-5361-4153-995e-8864ecd17cc8 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:10.554371 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a88cba-4d54-44bd-b846-77b9ad897254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:10.555362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a88cba-4d54-44bd-b846-77b9ad897254 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:10.555839 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a88cba-4d54-44bd-b846-77b9ad897254 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:10.556304 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a88cba-4d54-44bd-b846-77b9ad897254 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:10.557212 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a88cba-4d54-44bd-b846-77b9ad897254 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:10.558307 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a88cba-4d54-44bd-b846-77b9ad897254 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:10.559910 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a88cba-4d54-44bd-b846-77b9ad897254 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:10.567285 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149da335-c40e-4dee-bd45-0b597d25e843 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:10.567555 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149da335-c40e-4dee-bd45-0b597d25e843 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:10.567847 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149da335-c40e-4dee-bd45-0b597d25e843 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:10.569671 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 279/547] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:10.572820 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 269/548] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:10.574410 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27880c-a3d9-4f5b-b4a8-977654e44f50 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:10.574909 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27880c-a3d9-4f5b-b4a8-977654e44f50 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:10.575378 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27880c-a3d9-4f5b-b4a8-977654e44f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:10.578047 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a88cba-4d54-44bd-b846-77b9ad897254 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:10.617872 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-0b27880c-a3d9-4f5b-b4a8-977654e44f50 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:10.619403 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-0b27880c-a3d9-4f5b-b4a8-977654e44f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:10.642478 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:10.688483 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a88cba-4d54-44bd-b846-77b9ad897254 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:10.725275 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f00eef-4d33-4038-935f-670e696f88da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:10.726080 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f00eef-4d33-4038-935f-670e696f88da None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:10.726394 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f00eef-4d33-4038-935f-670e696f88da None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:10.726673 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f00eef-4d33-4038-935f-670e696f88da None admin] PATH_INFO: `/v3/users/swift` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:10.734798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f00eef-4d33-4038-935f-670e696f88da None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:10.736493 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f00eef-4d33-4038-935f-670e696f88da None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:10.751634 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-17f00eef-4d33-4038-935f-670e696f88da None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 19:25:10.751634 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:10.751634 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:10.751634 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:10.751634 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:10.751634 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:10.751634 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:10.751634 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:10.751634 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:10.751634 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:10.751634 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:10.751634 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:10.751634 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 19:25:10.751634 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 19:25:10.751634 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 19:25:10.751634 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 19:25:10.753262 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:10.753262 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:10.753262 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:25:10.753262 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:10.753262 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:25:10.753262 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:10.753262 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:10.753262 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:10.753262 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:10.753262 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:10.753262 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:10.753262 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:10.753262 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:10.753262 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:10.753262 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:10.753262 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:10.753262 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:10.753262 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:10.754598 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:10.754598 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:10.754598 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 19:25:10.754598 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 19:25:10.754598 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 19:25:10.754598 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 19:25:10.754598 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 19:25:10.754598 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 19:25:10.754598 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 19:25:10.754598 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:10.757424 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 270/549] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jun 15 19:25:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:25:10.758909 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f00eef-4d33-4038-935f-670e696f88da None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:10.769136 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:10.811876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 280/550] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:25:10.845812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27880c-a3d9-4f5b-b4a8-977654e44f50 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:10.874584 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:10.882643 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f00eef-4d33-4038-935f-670e696f88da None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:10.952295 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16449bb6-3a7c-4c9e-9aae-423294ab8844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:10.953901 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16449bb6-3a7c-4c9e-9aae-423294ab8844 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:10.954527 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16449bb6-3a7c-4c9e-9aae-423294ab8844 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:10.955948 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16449bb6-3a7c-4c9e-9aae-423294ab8844 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:10.957141 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16449bb6-3a7c-4c9e-9aae-423294ab8844 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:10.958381 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16449bb6-3a7c-4c9e-9aae-423294ab8844 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:10.960621 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16449bb6-3a7c-4c9e-9aae-423294ab8844 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:10.995779 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 271/551] 10.222.0.15 () {56 vars in 1135 bytes} [Wed Jun 15 19:25:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:25:11.002863 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16449bb6-3a7c-4c9e-9aae-423294ab8844 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:11.039124 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27880c-a3d9-4f5b-b4a8-977654e44f50 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:11.114991 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9770474f-afcd-4448-ad6e-6a95ad067064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:11.115801 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770474f-afcd-4448-ad6e-6a95ad067064 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:11.116087 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770474f-afcd-4448-ad6e-6a95ad067064 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:11.116357 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770474f-afcd-4448-ad6e-6a95ad067064 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:11.145910 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9770474f-afcd-4448-ad6e-6a95ad067064 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:25:11.153775 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-9770474f-afcd-4448-ad6e-6a95ad067064 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:25:11.153775 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:11.153775 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:11.153775 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:11.153775 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:11.153775 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:11.153775 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:11.153775 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:11.153775 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:11.153775 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:11.153775 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:11.153775 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:11.153775 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:25:11.153775 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:25:11.153775 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:25:11.153775 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:25:11.156426 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:25:11.156426 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:25:11.156426 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:25:11.156426 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:25:11.156426 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:11.156426 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:11.156426 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:11.156426 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:11.156426 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:11.156426 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:11.156426 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:11.156426 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:11.156426 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:11.156426 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:11.156426 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:11.156426 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:11.158656 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:11.158656 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:11.158656 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:11.158656 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:11.158656 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:25:11.158656 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:25:11.158656 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:25:11.158656 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:25:11.158656 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:25:11.158656 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:25:11.158656 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:25:11.158656 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:11.158656 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 281/552] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jun 15 19:25:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:25:11.162848 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9770474f-afcd-4448-ad6e-6a95ad067064 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:11.196735 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16449bb6-3a7c-4c9e-9aae-423294ab8844 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:11.237712 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3060b57f-c2ed-40a0-90a6-a32065d8414a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:11.238640 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3060b57f-c2ed-40a0-90a6-a32065d8414a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:11.238940 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3060b57f-c2ed-40a0-90a6-a32065d8414a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:11.239231 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3060b57f-c2ed-40a0-90a6-a32065d8414a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:11.264209 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:11.293445 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:11.331678 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3060b57f-c2ed-40a0-90a6-a32065d8414a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:11.333509 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3060b57f-c2ed-40a0-90a6-a32065d8414a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:11.365422 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9770474f-afcd-4448-ad6e-6a95ad067064 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:11.406256 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fbd9af-7126-4dbb-bcf4-b52290c067a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:11.407372 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fbd9af-7126-4dbb-bcf4-b52290c067a9 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:11.407853 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fbd9af-7126-4dbb-bcf4-b52290c067a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:11.408323 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fbd9af-7126-4dbb-bcf4-b52290c067a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:11.409303 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fbd9af-7126-4dbb-bcf4-b52290c067a9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:11.409820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fbd9af-7126-4dbb-bcf4-b52290c067a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:11.411662 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fbd9af-7126-4dbb-bcf4-b52290c067a9 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:11.421937 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 282/553] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jun 15 19:25:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:11.423676 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e75c30-01f3-4095-b220-a41ccadc5d38 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:11.424160 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e75c30-01f3-4095-b220-a41ccadc5d38 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:11.424628 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e75c30-01f3-4095-b220-a41ccadc5d38 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:11.426400 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 283/554] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:11.427318 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e75c30-01f3-4095-b220-a41ccadc5d38 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:11.455527 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 272/555] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:25:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 19:25:11.460440 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e87d6-c284-474b-ba3a-8c0e4e0fa9aa None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:11.460440 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e87d6-c284-474b-ba3a-8c0e4e0fa9aa None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:11.460440 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e87d6-c284-474b-ba3a-8c0e4e0fa9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:11.475414 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-8a8e87d6-c284-474b-ba3a-8c0e4e0fa9aa None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:11.476603 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8a8e87d6-c284-474b-ba3a-8c0e4e0fa9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:11.529156 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:11.547996 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e75c30-01f3-4095-b220-a41ccadc5d38 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:11.550864 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 273/556] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:25:11.552173 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19044e09-daa0-45dd-a116-e4d306d2af77 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:11.552466 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19044e09-daa0-45dd-a116-e4d306d2af77 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:11.552742 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19044e09-daa0-45dd-a116-e4d306d2af77 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:11.554209 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 274/557] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:11.554801 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19044e09-daa0-45dd-a116-e4d306d2af77 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:11.567939 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:11.596929 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89afa6a9-50c6-44a8-aaf2-7a004c1d2236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:11.597799 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afa6a9-50c6-44a8-aaf2-7a004c1d2236 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:11.598107 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afa6a9-50c6-44a8-aaf2-7a004c1d2236 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:11.598387 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afa6a9-50c6-44a8-aaf2-7a004c1d2236 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:11.609710 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89afa6a9-50c6-44a8-aaf2-7a004c1d2236 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:11.611367 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89afa6a9-50c6-44a8-aaf2-7a004c1d2236 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:11.631363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-89afa6a9-50c6-44a8-aaf2-7a004c1d2236 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:11.631363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:11.631363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:11.631363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:11.631363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:11.631363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:11.631363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:11.631363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:11.631363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:11.631363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:11.631363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:11.631363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:11.631363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:11.631363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:11.631363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:11.632423 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:11.632423 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:11.632423 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:11.632423 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:11.632423 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:11.632423 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:11.632423 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:11.632423 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:11.632423 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:11.632423 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:11.632423 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:11.632423 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:11.632423 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:11.632423 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:11.632423 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:11.632423 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:11.632423 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:11.633337 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:11.633337 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:11.633337 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:11.633337 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:11.633337 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 284/558] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:11.634480 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafcff3-181a-47c6-bce8-27972a7e6f8e None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:11.634768 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafcff3-181a-47c6-bce8-27972a7e6f8e None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:11.635046 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafcff3-181a-47c6-bce8-27972a7e6f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:11.652047 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-bbafcff3-181a-47c6-bce8-27972a7e6f8e None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:11.653231 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-bbafcff3-181a-47c6-bce8-27972a7e6f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:11.674872 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19044e09-daa0-45dd-a116-e4d306d2af77 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:11.717918 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a3f6a3-930d-4c3a-bc27-304e437f7d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:11.718754 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3f6a3-930d-4c3a-bc27-304e437f7d06 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:11.719040 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3f6a3-930d-4c3a-bc27-304e437f7d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:11.719315 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3f6a3-930d-4c3a-bc27-304e437f7d06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:11.734114 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 285/559] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:25:11.734114 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbafcff3-181a-47c6-bce8-27972a7e6f8e None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:11.738655 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a3f6a3-930d-4c3a-bc27-304e437f7d06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:11.740186 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a3f6a3-930d-4c3a-bc27-304e437f7d06 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:11.759842 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-67a3f6a3-930d-4c3a-bc27-304e437f7d06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:11.759842 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:11.759842 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:11.759842 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:11.759842 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:11.759842 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:11.759842 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:11.759842 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:11.759842 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:11.759842 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:11.759842 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:11.759842 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:11.759842 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:11.759842 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:11.759842 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:11.761269 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:11.761269 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:11.761269 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:11.761269 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:11.761269 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:11.761269 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:11.761269 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:11.761269 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:11.761269 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:11.761269 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:11.761269 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:11.761269 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:11.761269 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:11.761269 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:11.761269 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:11.761269 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:11.761269 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:11.762184 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:11.762184 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:11.762184 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:11.762184 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:11.762184 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 275/560] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:11.762184 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a3f6a3-930d-4c3a-bc27-304e437f7d06 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:11.796527 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:11.838136 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:11.847949 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbafcff3-181a-47c6-bce8-27972a7e6f8e None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:11.873347 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a3f6a3-930d-4c3a-bc27-304e437f7d06 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:11.884653 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae161a0-4450-4728-b639-0a9b1034825f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:11.886054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae161a0-4450-4728-b639-0a9b1034825f None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:11.886365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae161a0-4450-4728-b639-0a9b1034825f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:11.886636 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae161a0-4450-4728-b639-0a9b1034825f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:11.896676 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae161a0-4450-4728-b639-0a9b1034825f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:25:11.904377 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-fae161a0-4450-4728-b639-0a9b1034825f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:25:11.904377 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:11.904377 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:11.904377 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:11.904377 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:11.904377 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:11.904377 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:11.904377 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:11.904377 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:11.904377 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:11.904377 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:11.904377 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:11.904377 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:25:11.904377 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:25:11.904377 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:25:11.904377 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:25:11.905321 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:25:11.905321 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:25:11.905321 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:25:11.905321 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:25:11.905321 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:11.905321 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:11.905321 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:11.905321 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:11.905321 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:11.905321 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:11.905321 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:11.905321 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:11.905321 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:11.905321 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:11.905321 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:11.905321 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:11.906301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:11.906301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:11.906301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:11.906301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:11.906301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:25:11.906301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:25:11.906301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:25:11.906301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:25:11.906301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:25:11.906301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:25:11.906301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:25:11.906301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:11.908679 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4b610-d2bb-44ab-a983-c09295cc1615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:11.909337 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4b610-d2bb-44ab-a983-c09295cc1615 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:11.909615 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4b610-d2bb-44ab-a983-c09295cc1615 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:11.909883 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4b610-d2bb-44ab-a983-c09295cc1615 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:11.910399 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 286/561] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jun 15 19:25:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:11.910939 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a4b610-d2bb-44ab-a983-c09295cc1615 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:11.911242 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a4b610-d2bb-44ab-a983-c09295cc1615 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:11.911335 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae161a0-4450-4728-b639-0a9b1034825f None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:11.912650 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a4b610-d2bb-44ab-a983-c09295cc1615 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:11.923868 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 276/562] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:11.924648 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4b610-d2bb-44ab-a983-c09295cc1615 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:12.009377 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae161a0-4450-4728-b639-0a9b1034825f None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:12.016287 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:12.023418 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4b610-d2bb-44ab-a983-c09295cc1615 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:12.042803 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da216efc-0064-4523-ba50-8f842c1d301f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:12.043589 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da216efc-0064-4523-ba50-8f842c1d301f None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:12.043868 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da216efc-0064-4523-ba50-8f842c1d301f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:12.044139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da216efc-0064-4523-ba50-8f842c1d301f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:12.053871 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da216efc-0064-4523-ba50-8f842c1d301f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:25:12.056218 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad322c7-fa97-411e-b189-84ce9d17f885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:12.056866 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad322c7-fa97-411e-b189-84ce9d17f885 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:12.057145 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad322c7-fa97-411e-b189-84ce9d17f885 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:12.057915 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad322c7-fa97-411e-b189-84ce9d17f885 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:12.058660 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad322c7-fa97-411e-b189-84ce9d17f885 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:12.058964 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad322c7-fa97-411e-b189-84ce9d17f885 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:12.060321 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad322c7-fa97-411e-b189-84ce9d17f885 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:12.062022 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-da216efc-0064-4523-ba50-8f842c1d301f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:25:12.062022 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:12.062022 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:12.062022 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:12.062022 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:12.062022 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:12.062022 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:12.062022 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:12.062022 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:12.062022 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:12.062022 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:12.062022 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:12.062022 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:25:12.062022 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:25:12.062022 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:25:12.062022 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:25:12.063051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:25:12.063051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:25:12.063051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:25:12.063051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:25:12.063051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:12.063051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:12.063051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:12.063051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:12.063051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:12.063051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:12.063051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:12.063051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:12.063051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:12.063051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:12.063051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:12.063051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:12.063896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:12.063896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:12.063896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:12.063896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:12.063896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:25:12.063896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:25:12.063896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:25:12.063896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:25:12.063896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:25:12.063896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:25:12.063896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:25:12.063896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:12.063896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 287/563] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jun 15 19:25:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:12.064572 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4644e3b-1f16-4421-aa12-76a3568e6c02 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:12.064646 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4644e3b-1f16-4421-aa12-76a3568e6c02 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:12.065168 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4644e3b-1f16-4421-aa12-76a3568e6c02 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:12.066335 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 288/564] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:12.068284 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4644e3b-1f16-4421-aa12-76a3568e6c02 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:12.078829 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 277/565] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:12.078829 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad322c7-fa97-411e-b189-84ce9d17f885 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:12.085164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:12.170510 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4644e3b-1f16-4421-aa12-76a3568e6c02 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:12.171752 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad322c7-fa97-411e-b189-84ce9d17f885 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:12.203566 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b61c55-67aa-47db-bbce-0905aa0bfcdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:12.204222 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b61c55-67aa-47db-bbce-0905aa0bfcdb None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:12.204505 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b61c55-67aa-47db-bbce-0905aa0bfcdb None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:12.204779 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b61c55-67aa-47db-bbce-0905aa0bfcdb None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:12.205024 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129dab96-bcb7-4013-ade8-9bc21fb0c4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:12.205525 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b61c55-67aa-47db-bbce-0905aa0bfcdb None admin] RBAC: Adding query filter params (name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:12.205805 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129dab96-bcb7-4013-ade8-9bc21fb0c4a6 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:12.205907 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b61c55-67aa-47db-bbce-0905aa0bfcdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:12.206120 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129dab96-bcb7-4013-ade8-9bc21fb0c4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:12.206394 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129dab96-bcb7-4013-ade8-9bc21fb0c4a6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:12.207239 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b61c55-67aa-47db-bbce-0905aa0bfcdb None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:12.213389 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129dab96-bcb7-4013-ade8-9bc21fb0c4a6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:12.214872 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129dab96-bcb7-4013-ade8-9bc21fb0c4a6 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:12.215363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 289/566] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jun 15 19:25:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:12.216138 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b61c55-67aa-47db-bbce-0905aa0bfcdb None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:12.230386 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-129dab96-bcb7-4013-ade8-9bc21fb0c4a6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 19:25:12.230386 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:12.230386 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:12.230386 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:12.230386 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:12.230386 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:12.230386 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:12.230386 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:12.230386 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:12.230386 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:12.230386 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:12.230386 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:12.230386 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 19:25:12.230386 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 19:25:12.230386 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 19:25:12.230386 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 19:25:12.231426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:12.231426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:12.231426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:25:12.231426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:12.231426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:25:12.231426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:12.231426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:12.231426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:12.231426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:12.231426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:12.231426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:12.231426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:12.231426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:12.231426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:12.231426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:12.231426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:12.231426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:12.231426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:12.232362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:12.232362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:12.232362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 19:25:12.232362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 19:25:12.232362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 19:25:12.232362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 19:25:12.232362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 19:25:12.232362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 19:25:12.232362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 19:25:12.232362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:12.232362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 278/567] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Jun 15 19:25:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:12.232959 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4c63a-35c5-4b2f-9835-3e941c776ee9 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:12.233070 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4c63a-35c5-4b2f-9835-3e941c776ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:12.233339 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4c63a-35c5-4b2f-9835-3e941c776ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:12.249495 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-8fc4c63a-35c5-4b2f-9835-3e941c776ee9 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:12.250704 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8fc4c63a-35c5-4b2f-9835-3e941c776ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:12.260091 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:12.295282 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:12.316455 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 279/568] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:25:12.317266 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc4c63a-35c5-4b2f-9835-3e941c776ee9 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:12.334742 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b61c55-67aa-47db-bbce-0905aa0bfcdb None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:12.371409 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b4278e-c99d-45b7-ac7c-4d112f2a38b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:12.372071 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4278e-c99d-45b7-ac7c-4d112f2a38b5 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:12.372348 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4278e-c99d-45b7-ac7c-4d112f2a38b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:12.372621 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4278e-c99d-45b7-ac7c-4d112f2a38b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:12.373355 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b4278e-c99d-45b7-ac7c-4d112f2a38b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:12.373663 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b4278e-c99d-45b7-ac7c-4d112f2a38b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:12.375034 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b4278e-c99d-45b7-ac7c-4d112f2a38b5 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:12.382712 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 290/569] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jun 15 19:25:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:12.383745 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b4278e-c99d-45b7-ac7c-4d112f2a38b5 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:12.416430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc4c63a-35c5-4b2f-9835-3e941c776ee9 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:12.450990 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b3276-982e-4b88-a136-dfd1078f593a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:12.451777 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b3276-982e-4b88-a136-dfd1078f593a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:12.452058 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b3276-982e-4b88-a136-dfd1078f593a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:12.452331 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b3276-982e-4b88-a136-dfd1078f593a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:12.458975 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b3276-982e-4b88-a136-dfd1078f593a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:12.460540 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b3276-982e-4b88-a136-dfd1078f593a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:12.474988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-db9b3276-982e-4b88-a136-dfd1078f593a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:12.474988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:12.474988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:12.474988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:12.474988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:12.474988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:12.474988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:12.474988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:12.474988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:12.474988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:12.474988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:12.474988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:12.474988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 19:25:12.474988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 19:25:12.474988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 19:25:12.475873 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 19:25:12.475873 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:12.475873 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:12.475873 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:12.475873 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:12.475873 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:12.475873 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:12.475873 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:12.475873 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:12.475873 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:12.475873 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:12.475873 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:12.475873 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:12.475873 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:12.475873 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:12.475873 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:12.475873 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:12.476792 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 19:25:12.476792 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 19:25:12.476792 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:25:12.476792 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:25:12.476792 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:25:12.476792 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:25:12.476792 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:12.476792 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:12.476792 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 280/570] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:25:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:12.477271 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b3276-982e-4b88-a136-dfd1078f593a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:12.481731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:12.488832 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b4278e-c99d-45b7-ac7c-4d112f2a38b5 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:12.524750 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c131f1aa-3779-4bc3-8567-8e969ae9a9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:12.525463 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131f1aa-3779-4bc3-8567-8e969ae9a9c0 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:12.525746 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131f1aa-3779-4bc3-8567-8e969ae9a9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:12.526032 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131f1aa-3779-4bc3-8567-8e969ae9a9c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:12.530852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:12.539238 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c131f1aa-3779-4bc3-8567-8e969ae9a9c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:12.540704 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c131f1aa-3779-4bc3-8567-8e969ae9a9c0 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:12.558356 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-c131f1aa-3779-4bc3-8567-8e969ae9a9c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:12.558356 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:12.558356 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:12.558356 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:12.558356 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:12.558356 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:12.558356 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:12.558356 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:12.558356 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:12.558356 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:12.558356 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:12.558356 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:12.558356 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:12.558356 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:12.558356 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:12.559275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:12.559275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:12.559275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:12.559275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:12.559275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:12.559275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:12.559275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:12.559275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:12.559275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:12.559275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:12.559275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:12.559275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:12.559275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:12.559275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:12.559275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:12.559275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:12.559275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:12.560137 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:12.560137 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:12.560137 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:12.560137 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:12.560137 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 291/571] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:12.560414 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c131f1aa-3779-4bc3-8567-8e969ae9a9c0 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:12.594762 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b3276-982e-4b88-a136-dfd1078f593a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:12.627920 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a413fa-0557-4323-ae40-8f2d9a33f9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:12.628584 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a413fa-0557-4323-ae40-8f2d9a33f9f1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:12.628870 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a413fa-0557-4323-ae40-8f2d9a33f9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:12.629146 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a413fa-0557-4323-ae40-8f2d9a33f9f1 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:12.629985 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a413fa-0557-4323-ae40-8f2d9a33f9f1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:12.630325 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a413fa-0557-4323-ae40-8f2d9a33f9f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:12.631758 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a413fa-0557-4323-ae40-8f2d9a33f9f1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:12.648507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 281/572] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Jun 15 19:25:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:12.649290 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a413fa-0557-4323-ae40-8f2d9a33f9f1 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:12.658059 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c131f1aa-3779-4bc3-8567-8e969ae9a9c0 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:12.690829 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1144a0-17e2-4e77-abe3-b023a3e5ebf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:12.691619 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1144a0-17e2-4e77-abe3-b023a3e5ebf2 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:12.691898 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1144a0-17e2-4e77-abe3-b023a3e5ebf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:12.692168 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1144a0-17e2-4e77-abe3-b023a3e5ebf2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:12.702077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1144a0-17e2-4e77-abe3-b023a3e5ebf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:25:12.709702 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-3c1144a0-17e2-4e77-abe3-b023a3e5ebf2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:25:12.709702 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:12.709702 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:12.709702 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:12.709702 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:12.709702 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:12.709702 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:12.709702 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:12.709702 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:12.709702 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:12.709702 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:12.709702 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:12.709702 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:25:12.709702 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:25:12.709702 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:25:12.709702 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:25:12.710720 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:25:12.710720 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:25:12.710720 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:25:12.710720 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:25:12.710720 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:12.710720 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:12.710720 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:12.710720 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:12.710720 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:12.710720 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:12.710720 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:12.710720 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:12.710720 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:12.710720 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:12.710720 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:12.710720 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:12.711614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:12.711614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:12.711614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:12.711614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:12.711614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:25:12.711614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:25:12.711614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:25:12.711614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:25:12.711614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:25:12.711614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:25:12.711614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:25:12.711614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:12.712256 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:12.712330 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 292/573] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jun 15 19:25:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:12.712330 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1144a0-17e2-4e77-abe3-b023a3e5ebf2 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:12.754206 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a413fa-0557-4323-ae40-8f2d9a33f9f1 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:12.759714 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:12.789054 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b997df5f-7d3c-4aa5-88ef-53c7730f01ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:12.789775 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997df5f-7d3c-4aa5-88ef-53c7730f01ad None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:12.790056 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997df5f-7d3c-4aa5-88ef-53c7730f01ad None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:12.790356 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997df5f-7d3c-4aa5-88ef-53c7730f01ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:12.800260 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b997df5f-7d3c-4aa5-88ef-53c7730f01ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:12.801696 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b997df5f-7d3c-4aa5-88ef-53c7730f01ad None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:12.811986 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1144a0-17e2-4e77-abe3-b023a3e5ebf2 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:12.821657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-b997df5f-7d3c-4aa5-88ef-53c7730f01ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:12.821657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:12.821657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:12.821657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:12.821657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:12.821657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:12.821657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:12.821657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:12.821657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:12.821657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:12.821657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:12.821657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:12.821657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:12.821657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:12.821657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:12.822800 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:12.822800 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:12.822800 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:12.822800 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:12.822800 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:12.822800 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:12.822800 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:12.822800 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:12.822800 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:12.822800 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:12.822800 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:12.822800 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:12.822800 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:12.822800 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:12.822800 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:12.822800 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:12.822800 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:12.824370 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:12.824370 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:12.824370 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:12.824370 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:12.824370 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 282/574] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:12.824370 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b997df5f-7d3c-4aa5-88ef-53c7730f01ad None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:12.847812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ab5ff-87b9-4a4a-8484-ed45d60738c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:12.848470 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ab5ff-87b9-4a4a-8484-ed45d60738c1 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:12.848750 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ab5ff-87b9-4a4a-8484-ed45d60738c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:12.849027 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ab5ff-87b9-4a4a-8484-ed45d60738c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:12.849808 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782ab5ff-87b9-4a4a-8484-ed45d60738c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:12.850143 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782ab5ff-87b9-4a4a-8484-ed45d60738c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:12.851580 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782ab5ff-87b9-4a4a-8484-ed45d60738c1 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:12.863759 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 293/575] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jun 15 19:25:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:12.864527 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ab5ff-87b9-4a4a-8484-ed45d60738c1 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:12.923553 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b997df5f-7d3c-4aa5-88ef-53c7730f01ad None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:12.937501 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:12.957032 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabfad38-c82c-4e3e-a347-c048dd5a91fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:12.957685 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabfad38-c82c-4e3e-a347-c048dd5a91fc None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:12.957965 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabfad38-c82c-4e3e-a347-c048dd5a91fc None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:12.958265 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabfad38-c82c-4e3e-a347-c048dd5a91fc None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:12.958980 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabfad38-c82c-4e3e-a347-c048dd5a91fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:12.959284 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabfad38-c82c-4e3e-a347-c048dd5a91fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:12.960641 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabfad38-c82c-4e3e-a347-c048dd5a91fc None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:12.960819 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ab5ff-87b9-4a4a-8484-ed45d60738c1 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:12.970061 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:12.971599 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 283/576] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:12.972387 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabfad38-c82c-4e3e-a347-c048dd5a91fc None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:12.997414 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674218c6-b15f-4082-84a2-46bf17cb7e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:12.998226 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674218c6-b15f-4082-84a2-46bf17cb7e4a None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:12.998506 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674218c6-b15f-4082-84a2-46bf17cb7e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:12.998779 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674218c6-b15f-4082-84a2-46bf17cb7e4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:13.008827 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674218c6-b15f-4082-84a2-46bf17cb7e4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:13.010294 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674218c6-b15f-4082-84a2-46bf17cb7e4a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:13.027934 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-674218c6-b15f-4082-84a2-46bf17cb7e4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:13.027934 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:13.027934 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:13.027934 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:13.027934 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:13.027934 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:13.027934 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:13.027934 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:13.027934 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:13.027934 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:13.027934 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:13.027934 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:13.027934 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:13.027934 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:13.027934 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:13.028884 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:13.028884 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:13.028884 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:13.028884 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:13.028884 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:13.028884 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:13.028884 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:13.028884 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:13.028884 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:13.028884 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:13.028884 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:13.028884 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:13.028884 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:13.028884 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:13.028884 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:13.028884 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:13.028884 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:13.029778 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:13.029778 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:13.029778 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:13.029778 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:13.029778 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 294/577] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:13.030456 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674218c6-b15f-4082-84a2-46bf17cb7e4a None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:13.075447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabfad38-c82c-4e3e-a347-c048dd5a91fc None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:13.110320 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8fe6e2-ecaa-4a06-9db2-35d4e9d8cf2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:13.110990 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8fe6e2-ecaa-4a06-9db2-35d4e9d8cf2b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:13.111278 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8fe6e2-ecaa-4a06-9db2-35d4e9d8cf2b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:13.111548 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8fe6e2-ecaa-4a06-9db2-35d4e9d8cf2b None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:13.112290 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8fe6e2-ecaa-4a06-9db2-35d4e9d8cf2b None admin] RBAC: Adding query filter params (name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:13.112593 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8fe6e2-ecaa-4a06-9db2-35d4e9d8cf2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:13.114351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8fe6e2-ecaa-4a06-9db2-35d4e9d8cf2b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:13.125573 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 284/578] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jun 15 19:25:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:13.126330 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8fe6e2-ecaa-4a06-9db2-35d4e9d8cf2b None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:13.135864 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674218c6-b15f-4082-84a2-46bf17cb7e4a None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:13.164138 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:13.170856 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed318f49-dec7-405b-b542-7bc2e7b433b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:13.171520 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed318f49-dec7-405b-b542-7bc2e7b433b1 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:13.171803 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed318f49-dec7-405b-b542-7bc2e7b433b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:13.172080 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed318f49-dec7-405b-b542-7bc2e7b433b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:13.172797 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed318f49-dec7-405b-b542-7bc2e7b433b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:13.173104 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed318f49-dec7-405b-b542-7bc2e7b433b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:13.174516 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed318f49-dec7-405b-b542-7bc2e7b433b1 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:13.186877 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 295/579] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:13.187469 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed318f49-dec7-405b-b542-7bc2e7b433b1 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:13.207634 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:13.233953 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8fe6e2-ecaa-4a06-9db2-35d4e9d8cf2b None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:13.265946 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732517af-ca45-4d96-95bd-a42880451084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:13.266617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732517af-ca45-4d96-95bd-a42880451084 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:13.266899 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732517af-ca45-4d96-95bd-a42880451084 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:13.267177 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732517af-ca45-4d96-95bd-a42880451084 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:13.268023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732517af-ca45-4d96-95bd-a42880451084 None admin] RBAC: Adding query filter params (role.id=a8df9714563b435e9fce55a04e74ae5e, scope.project.id=7fb8cc0eb560465e972ffd45867ee73b, user.id=176d5775a70f4524a9b7fe1ce9d426ed) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:13.268329 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732517af-ca45-4d96-95bd-a42880451084 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:13.269749 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732517af-ca45-4d96-95bd-a42880451084 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:13.275101 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 285/580] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:25:13 2022] GET /identity/v3/role_assignments?user.id=176d5775a70f4524a9b7fe1ce9d426ed&scope.project.id=7fb8cc0eb560465e972ffd45867ee73b&role.id=a8df9714563b435e9fce55a04e74ae5e => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:13.276391 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732517af-ca45-4d96-95bd-a42880451084 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:13.286328 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed318f49-dec7-405b-b542-7bc2e7b433b1 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:13.324850 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8bf1f5-d7df-4b70-9c3a-bc82fbf5f105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:13.325744 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8bf1f5-d7df-4b70-9c3a-bc82fbf5f105 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:13.325960 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8bf1f5-d7df-4b70-9c3a-bc82fbf5f105 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:13.326267 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8bf1f5-d7df-4b70-9c3a-bc82fbf5f105 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:13.334837 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8bf1f5-d7df-4b70-9c3a-bc82fbf5f105 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:13.336440 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8bf1f5-d7df-4b70-9c3a-bc82fbf5f105 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:13.352836 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-bd8bf1f5-d7df-4b70-9c3a-bc82fbf5f105 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 15 19:25:13.352836 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:13.352836 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:13.352836 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:13.352836 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:13.352836 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:13.352836 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:13.352836 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:13.352836 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:13.352836 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:13.352836 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:13.352836 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:13.352836 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 19:25:13.352836 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 19:25:13.352836 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 19:25:13.352836 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 19:25:13.353876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:13.353876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:13.353876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:25:13.353876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:13.353876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:25:13.353876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:13.353876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:13.353876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:13.353876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:13.353876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:13.353876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:13.353876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:13.353876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:13.353876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:13.353876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:13.353876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:13.353876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:13.353876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:13.355468 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:13.355468 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:13.355468 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 19:25:13.355468 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 19:25:13.355468 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 19:25:13.355468 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 19:25:13.355468 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 19:25:13.355468 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 19:25:13.355468 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 15 19:25:13.355468 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:13.355468 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 296/581] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:25:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:13.355468 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8bf1f5-d7df-4b70-9c3a-bc82fbf5f105 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:13.416106 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:13.420715 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732517af-ca45-4d96-95bd-a42880451084 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:13.447604 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:13.462716 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf3c6d-0384-498b-853b-d07ec6ba6bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:13.463433 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf3c6d-0384-498b-853b-d07ec6ba6bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:13.463703 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf3c6d-0384-498b-853b-d07ec6ba6bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:13.463962 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf3c6d-0384-498b-853b-d07ec6ba6bd4 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:13.464749 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf3c6d-0384-498b-853b-d07ec6ba6bd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:13.464992 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf3c6d-0384-498b-853b-d07ec6ba6bd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:13.466399 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf3c6d-0384-498b-853b-d07ec6ba6bd4 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:13.471729 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8bf1f5-d7df-4b70-9c3a-bc82fbf5f105 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:13.483080 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 286/582] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:13.483080 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf3c6d-0384-498b-853b-d07ec6ba6bd4 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:13.513427 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2682741-e104-4515-9fa7-e5e8d8ffef2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:13.514335 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2682741-e104-4515-9fa7-e5e8d8ffef2d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:13.514691 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2682741-e104-4515-9fa7-e5e8d8ffef2d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:13.515031 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2682741-e104-4515-9fa7-e5e8d8ffef2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:13.526499 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2682741-e104-4515-9fa7-e5e8d8ffef2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:13.528023 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2682741-e104-4515-9fa7-e5e8d8ffef2d None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:13.545648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-c2682741-e104-4515-9fa7-e5e8d8ffef2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:13.545648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:13.545648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:13.545648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:13.545648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:13.545648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:13.545648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:13.545648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:13.545648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:13.545648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:13.545648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:13.545648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:13.545648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:13.545648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:13.545648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:13.546728 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:13.546728 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:13.546728 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:13.546728 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:13.546728 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:13.546728 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:13.546728 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:13.546728 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:13.546728 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:13.546728 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:13.546728 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:13.546728 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:13.546728 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:13.546728 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:13.546728 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:13.546728 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:13.546728 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:13.547851 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:13.547851 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:13.547851 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:13.547851 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:13.553045 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 297/583] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:13.553045 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2682741-e104-4515-9fa7-e5e8d8ffef2d None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:13.597985 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf3c6d-0384-498b-853b-d07ec6ba6bd4 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:13.634272 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a09da-b8e2-4105-b47b-4b878c469a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:13.635073 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a09da-b8e2-4105-b47b-4b878c469a7c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:13.635888 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a09da-b8e2-4105-b47b-4b878c469a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:13.636167 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a09da-b8e2-4105-b47b-4b878c469a7c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:13.643640 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5a09da-b8e2-4105-b47b-4b878c469a7c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:13.645103 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5a09da-b8e2-4105-b47b-4b878c469a7c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:13.651641 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2682741-e104-4515-9fa7-e5e8d8ffef2d None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:13.659506 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-cc5a09da-b8e2-4105-b47b-4b878c469a7c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 19:25:13.659506 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:13.659506 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:13.659506 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:13.659506 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:13.659506 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:13.659506 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:13.659506 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:13.659506 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:13.659506 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:13.659506 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:13.659506 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:13.659506 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 19:25:13.659506 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 19:25:13.659506 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 19:25:13.659506 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 19:25:13.660451 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:13.660451 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:13.660451 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:25:13.660451 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:13.660451 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:25:13.660451 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:13.660451 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:13.660451 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:13.660451 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:13.660451 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:13.660451 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:13.660451 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:13.660451 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:13.660451 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:13.660451 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:13.660451 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:13.660451 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:13.660451 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:13.661375 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:13.661375 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:13.661375 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 19:25:13.661375 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 19:25:13.661375 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 19:25:13.661375 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 19:25:13.661375 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 19:25:13.661375 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 19:25:13.661375 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 19:25:13.661375 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:13.661375 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 287/584] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:25:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:13.661950 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a09da-b8e2-4105-b47b-4b878c469a7c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:13.684384 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:13.687255 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:13.692736 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb24cdf5-529a-4053-8f18-019507b82312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:13.693410 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb24cdf5-529a-4053-8f18-019507b82312 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:13.693707 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb24cdf5-529a-4053-8f18-019507b82312 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:13.693969 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb24cdf5-529a-4053-8f18-019507b82312 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:13.694829 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb24cdf5-529a-4053-8f18-019507b82312 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:13.695135 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb24cdf5-529a-4053-8f18-019507b82312 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:13.696558 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb24cdf5-529a-4053-8f18-019507b82312 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:13.719169 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 298/585] 10.222.0.15 () {56 vars in 1137 bytes} [Wed Jun 15 19:25:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:13.719916 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb24cdf5-529a-4053-8f18-019507b82312 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:13.780856 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a09da-b8e2-4105-b47b-4b878c469a7c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:13.820582 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb24cdf5-529a-4053-8f18-019507b82312 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:13.825452 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b190cbe-a994-4166-9e1f-667a5466b534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:13.826280 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b190cbe-a994-4166-9e1f-667a5466b534 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:13.826557 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b190cbe-a994-4166-9e1f-667a5466b534 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:13.826835 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b190cbe-a994-4166-9e1f-667a5466b534 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:13.833808 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b190cbe-a994-4166-9e1f-667a5466b534 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:13.835185 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b190cbe-a994-4166-9e1f-667a5466b534 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:13.849400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-4b190cbe-a994-4166-9e1f-667a5466b534 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:13.849400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:13.849400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:13.849400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:13.849400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:13.849400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:13.849400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:13.849400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:13.849400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:13.849400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:13.849400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:13.849400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:13.849400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 19:25:13.849400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 19:25:13.849400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 19:25:13.850574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 19:25:13.850574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:13.850574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:13.850574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:13.850574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:13.850574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:13.850574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:13.850574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:13.850574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:13.850574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:13.850574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:13.850574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:13.850574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:13.850574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:13.850574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:13.850574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:13.850574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:13.851447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 19:25:13.851447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 19:25:13.851447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:25:13.851447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:25:13.851447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:25:13.851447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:25:13.851447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:13.851447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:13.851447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 288/586] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:25:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:13.851447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b190cbe-a994-4166-9e1f-667a5466b534 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:13.862820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c662066-3bb0-46d2-8ab2-88d09ae62355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:13.863471 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c662066-3bb0-46d2-8ab2-88d09ae62355 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:13.863762 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c662066-3bb0-46d2-8ab2-88d09ae62355 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:13.864022 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c662066-3bb0-46d2-8ab2-88d09ae62355 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:13.864742 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c662066-3bb0-46d2-8ab2-88d09ae62355 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:13.865156 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c662066-3bb0-46d2-8ab2-88d09ae62355 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:13.866419 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c662066-3bb0-46d2-8ab2-88d09ae62355 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:13.878589 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 299/587] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:13.878589 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c662066-3bb0-46d2-8ab2-88d09ae62355 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:13.935558 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:13.942316 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:13.963814 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b190cbe-a994-4166-9e1f-667a5466b534 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:13.997110 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c662066-3bb0-46d2-8ab2-88d09ae62355 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:13.999715 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e853f-d5b3-44a8-9601-cc6f3830030f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:14.000393 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e853f-d5b3-44a8-9601-cc6f3830030f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:14.000676 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e853f-d5b3-44a8-9601-cc6f3830030f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:14.000948 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e853f-d5b3-44a8-9601-cc6f3830030f None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:14.001787 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548e853f-d5b3-44a8-9601-cc6f3830030f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:14.002115 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548e853f-d5b3-44a8-9601-cc6f3830030f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:14.003558 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548e853f-d5b3-44a8-9601-cc6f3830030f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:14.021723 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 289/588] 10.222.0.15 () {56 vars in 1137 bytes} [Wed Jun 15 19:25:13 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:14.027159 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e853f-d5b3-44a8-9601-cc6f3830030f None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:14.033316 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561ae8f2-2d2e-4c03-b099-872c77f35a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:14.034210 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ae8f2-2d2e-4c03-b099-872c77f35a85 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:14.034557 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ae8f2-2d2e-4c03-b099-872c77f35a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:14.034895 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ae8f2-2d2e-4c03-b099-872c77f35a85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:14.045747 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561ae8f2-2d2e-4c03-b099-872c77f35a85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:14.047408 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561ae8f2-2d2e-4c03-b099-872c77f35a85 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:14.065002 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-561ae8f2-2d2e-4c03-b099-872c77f35a85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:14.065002 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:14.065002 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:14.065002 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:14.065002 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:14.065002 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:14.065002 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:14.065002 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:14.065002 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:14.065002 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:14.065002 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:14.065002 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:14.065002 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:14.065002 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:14.065002 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:14.066106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:14.066106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:14.066106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:14.066106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:14.066106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:14.066106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:14.066106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:14.066106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:14.066106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:14.066106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:14.066106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:14.066106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:14.066106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:14.066106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:14.066106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:14.066106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:14.066106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:14.067193 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:14.067193 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:14.067193 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:14.067193 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:14.070787 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 300/589] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:14.071485 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561ae8f2-2d2e-4c03-b099-872c77f35a85 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:14.139872 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e853f-d5b3-44a8-9601-cc6f3830030f None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:14.176748 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc9741f-f799-4369-91ff-649273171075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:14.177425 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9741f-f799-4369-91ff-649273171075 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:14.177708 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9741f-f799-4369-91ff-649273171075 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:14.177981 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9741f-f799-4369-91ff-649273171075 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:14.178794 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc9741f-f799-4369-91ff-649273171075 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:14.179102 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc9741f-f799-4369-91ff-649273171075 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:14.179192 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561ae8f2-2d2e-4c03-b099-872c77f35a85 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:14.180525 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc9741f-f799-4369-91ff-649273171075 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:14.181008 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:14.188348 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:14.198813 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 290/590] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jun 15 19:25:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:14.198813 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc9741f-f799-4369-91ff-649273171075 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:14.227043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d3faf6-ab6a-44ca-9501-952e51990e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:14.227841 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3faf6-ab6a-44ca-9501-952e51990e14 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:14.228123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3faf6-ab6a-44ca-9501-952e51990e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:14.228399 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3faf6-ab6a-44ca-9501-952e51990e14 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:14.236024 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d3faf6-ab6a-44ca-9501-952e51990e14 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:14.237460 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d3faf6-ab6a-44ca-9501-952e51990e14 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:14.252015 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-72d3faf6-ab6a-44ca-9501-952e51990e14 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 19:25:14.252015 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:14.252015 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:14.252015 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:14.252015 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:14.252015 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:14.252015 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:14.252015 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:14.252015 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:14.252015 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:14.252015 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:14.252015 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:14.252015 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 19:25:14.252015 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 19:25:14.252015 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 19:25:14.252015 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 19:25:14.252938 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:14.252938 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:14.252938 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:25:14.252938 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:14.252938 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:25:14.252938 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:14.252938 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:14.252938 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:14.252938 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:14.252938 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:14.252938 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:14.252938 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:14.252938 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:14.252938 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:14.252938 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:14.252938 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:14.252938 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:14.252938 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:14.253910 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:14.253910 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:14.253910 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 19:25:14.253910 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 19:25:14.253910 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 19:25:14.253910 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 19:25:14.253910 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 19:25:14.253910 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 19:25:14.253910 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 19:25:14.253910 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:14.253910 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 301/591] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jun 15 19:25:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:14.254524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d3faf6-ab6a-44ca-9501-952e51990e14 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:14.305161 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc9741f-f799-4369-91ff-649273171075 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:14.351448 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d32e4-d846-4dec-a815-1be7bc4ade6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:14.353169 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d32e4-d846-4dec-a815-1be7bc4ade6f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:14.353745 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d32e4-d846-4dec-a815-1be7bc4ade6f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:14.354789 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d32e4-d846-4dec-a815-1be7bc4ade6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:14.359548 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d3faf6-ab6a-44ca-9501-952e51990e14 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:14.365511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9d32e4-d846-4dec-a815-1be7bc4ade6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:14.366998 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9d32e4-d846-4dec-a815-1be7bc4ade6f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:14.384628 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-3e9d32e4-d846-4dec-a815-1be7bc4ade6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:14.384628 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:14.384628 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:14.384628 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:14.384628 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:14.384628 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:14.384628 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:14.384628 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:14.384628 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:14.384628 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:14.384628 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:14.384628 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:14.384628 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:14.384628 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:14.384628 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:14.386286 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:14.386286 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:14.386286 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:14.386286 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:14.386286 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:14.386286 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:14.386286 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:14.386286 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:14.386286 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:14.386286 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:14.386286 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:14.386286 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:14.386286 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:14.386286 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:14.386286 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:14.386286 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:14.386286 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:14.387395 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:14.387395 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:14.387395 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:14.387395 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:14.387395 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 291/592] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:14.387395 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d32e4-d846-4dec-a815-1be7bc4ade6f None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:14.399066 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d9a1b-bd9f-404f-8ad7-446b54cc1a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:14.399658 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d9a1b-bd9f-404f-8ad7-446b54cc1a31 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:14.399939 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d9a1b-bd9f-404f-8ad7-446b54cc1a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:14.400240 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d9a1b-bd9f-404f-8ad7-446b54cc1a31 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:14.401086 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781d9a1b-bd9f-404f-8ad7-446b54cc1a31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:14.401394 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781d9a1b-bd9f-404f-8ad7-446b54cc1a31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:14.402768 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781d9a1b-bd9f-404f-8ad7-446b54cc1a31 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:14.415549 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 302/593] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:14.416349 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d9a1b-bd9f-404f-8ad7-446b54cc1a31 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:14.441971 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:14.443722 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:14.496964 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d32e4-d846-4dec-a815-1be7bc4ade6f None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:14.523925 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d9a1b-bd9f-404f-8ad7-446b54cc1a31 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:14.533696 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e658e7-14cc-4282-a3e1-5146ca7eeb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:14.535081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e658e7-14cc-4282-a3e1-5146ca7eeb92 None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:14.535365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e658e7-14cc-4282-a3e1-5146ca7eeb92 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:14.535904 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e658e7-14cc-4282-a3e1-5146ca7eeb92 None admin] PATH_INFO: `/v3/projects/7fb8cc0eb560465e972ffd45867ee73b/users/3e74ac44e5814122a3226ac023f3688a/roles/a8df9714563b435e9fce55a04e74ae5e` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:14.551615 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e658e7-14cc-4282-a3e1-5146ca7eeb92 None admin] RBAC: Authorizing `identity:create_grant(project_id=7fb8cc0eb560465e972ffd45867ee73b, user_id=3e74ac44e5814122a3226ac023f3688a, role_id=a8df9714563b435e9fce55a04e74ae5e)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:14.553071 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e658e7-14cc-4282-a3e1-5146ca7eeb92 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:14.560961 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845e70d0-ebb9-4d07-a7a1-36d201413fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:14.562145 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e70d0-ebb9-4d07-a7a1-36d201413fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:14.562440 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e70d0-ebb9-4d07-a7a1-36d201413fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:14.562867 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e70d0-ebb9-4d07-a7a1-36d201413fe3 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:14.563831 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845e70d0-ebb9-4d07-a7a1-36d201413fe3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:14.564147 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845e70d0-ebb9-4d07-a7a1-36d201413fe3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:14.565589 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845e70d0-ebb9-4d07-a7a1-36d201413fe3 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:14.574359 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 292/594] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jun 15 19:25:14 2022] PUT /identity/v3/projects/7fb8cc0eb560465e972ffd45867ee73b/users/3e74ac44e5814122a3226ac023f3688a/roles/a8df9714563b435e9fce55a04e74ae5e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:25:14.577199 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e658e7-14cc-4282-a3e1-5146ca7eeb92 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:14.594691 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 303/595] 10.222.0.15 () {56 vars in 1139 bytes} [Wed Jun 15 19:25:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:14.595920 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845e70d0-ebb9-4d07-a7a1-36d201413fe3 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:14.708153 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:14.715063 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:14.721135 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e658e7-14cc-4282-a3e1-5146ca7eeb92 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:14.741582 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845e70d0-ebb9-4d07-a7a1-36d201413fe3 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:14.773228 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b215d63e-c018-4ba0-8d25-2e7a0540ec3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:14.774458 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215d63e-c018-4ba0-8d25-2e7a0540ec3d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:14.775137 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215d63e-c018-4ba0-8d25-2e7a0540ec3d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:14.775560 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215d63e-c018-4ba0-8d25-2e7a0540ec3d None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:14.777753 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b215d63e-c018-4ba0-8d25-2e7a0540ec3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:14.778252 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b215d63e-c018-4ba0-8d25-2e7a0540ec3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:14.780759 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b215d63e-c018-4ba0-8d25-2e7a0540ec3d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:14.785382 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4c7fa5-24b4-4269-95c4-71e904a7abc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:14.786372 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c7fa5-24b4-4269-95c4-71e904a7abc4 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:14.786784 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c7fa5-24b4-4269-95c4-71e904a7abc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:14.787138 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c7fa5-24b4-4269-95c4-71e904a7abc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:14.794859 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 293/596] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:14.796603 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4c7fa5-24b4-4269-95c4-71e904a7abc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:14.796770 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b215d63e-c018-4ba0-8d25-2e7a0540ec3d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:14.798281 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4c7fa5-24b4-4269-95c4-71e904a7abc4 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:14.818359 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-4c4c7fa5-24b4-4269-95c4-71e904a7abc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:14.818359 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:14.818359 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:14.818359 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:14.818359 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:14.818359 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:14.818359 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:14.818359 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:14.818359 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:14.818359 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:14.818359 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:14.818359 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:14.818359 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 19:25:14.818359 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 19:25:14.818359 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 19:25:14.819320 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 19:25:14.819320 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:14.819320 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:14.819320 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:14.819320 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:14.819320 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:14.819320 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:14.819320 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:14.819320 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:14.819320 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:14.819320 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:14.819320 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:14.819320 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:14.819320 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:14.819320 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:14.819320 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:14.819320 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:14.820504 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 19:25:14.820504 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 19:25:14.820504 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:25:14.820504 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:25:14.820504 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:25:14.820504 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:25:14.820504 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:14.820504 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:14.820504 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 304/597] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:25:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:14.821378 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4c7fa5-24b4-4269-95c4-71e904a7abc4 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:14.933435 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b215d63e-c018-4ba0-8d25-2e7a0540ec3d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:14.960388 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4c7fa5-24b4-4269-95c4-71e904a7abc4 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:14.978275 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24df5e7-f5eb-452c-86c2-e68092e395d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:14.980063 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24df5e7-f5eb-452c-86c2-e68092e395d2 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:14.980494 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24df5e7-f5eb-452c-86c2-e68092e395d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:14.980934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24df5e7-f5eb-452c-86c2-e68092e395d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:14.995863 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24df5e7-f5eb-452c-86c2-e68092e395d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:14.998529 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24df5e7-f5eb-452c-86c2-e68092e395d2 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:15.006038 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd46ea7-34a2-42a4-ac95-128dc088fc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:15.007496 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd46ea7-34a2-42a4-ac95-128dc088fc70 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:15.007884 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd46ea7-34a2-42a4-ac95-128dc088fc70 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:15.008265 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd46ea7-34a2-42a4-ac95-128dc088fc70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:15.016847 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd46ea7-34a2-42a4-ac95-128dc088fc70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:15.018503 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd46ea7-34a2-42a4-ac95-128dc088fc70 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:15.019774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-d24df5e7-f5eb-452c-86c2-e68092e395d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:15.019774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:15.019774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:15.019774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:15.019774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:15.019774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:15.019774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:15.019774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:15.019774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:15.019774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:15.019774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:15.019774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:15.019774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:15.019774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:15.019774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:15.020790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:15.020790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:15.020790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:15.020790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:15.020790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:15.020790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:15.020790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:15.020790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:15.020790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:15.020790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:15.020790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:15.020790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:15.020790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:15.020790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:15.020790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:15.020790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:15.020790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:15.021676 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:15.021676 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:15.021676 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:15.021676 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:15.023590 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:15.025087 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 294/598] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:15.026803 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24df5e7-f5eb-452c-86c2-e68092e395d2 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:15.042123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-bbd46ea7-34a2-42a4-ac95-128dc088fc70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:15.042123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:15.042123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:15.042123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:15.042123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:15.042123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:15.042123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:15.042123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:15.042123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:15.042123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:15.042123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:15.042123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:15.042123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 19:25:15.042123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 19:25:15.042123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 19:25:15.043202 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 19:25:15.043202 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:15.043202 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:15.043202 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:15.043202 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:15.043202 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:15.043202 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:15.043202 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:15.043202 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:15.043202 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:15.043202 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:15.043202 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:15.043202 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:15.043202 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:15.043202 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:15.043202 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:15.043202 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:15.044068 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 19:25:15.044068 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 19:25:15.044068 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:25:15.044068 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:25:15.044068 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:25:15.044068 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:25:15.044068 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:15.044068 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:15.044663 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 305/599] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:25:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:15.045746 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd46ea7-34a2-42a4-ac95-128dc088fc70 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:15.047199 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:15.152051 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24df5e7-f5eb-452c-86c2-e68092e395d2 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:15.179766 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd46ea7-34a2-42a4-ac95-128dc088fc70 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:15.198716 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e56bb75-98a2-48e8-8dbc-d4b950a2936c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:15.199654 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56bb75-98a2-48e8-8dbc-d4b950a2936c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:15.200111 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56bb75-98a2-48e8-8dbc-d4b950a2936c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:15.200584 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56bb75-98a2-48e8-8dbc-d4b950a2936c None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:15.202108 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e56bb75-98a2-48e8-8dbc-d4b950a2936c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:15.202600 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e56bb75-98a2-48e8-8dbc-d4b950a2936c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:15.204736 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e56bb75-98a2-48e8-8dbc-d4b950a2936c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:15.218768 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35545a-a9e4-4a98-9edc-d828ec2b8c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:15.219751 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35545a-a9e4-4a98-9edc-d828ec2b8c09 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:15.220195 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35545a-a9e4-4a98-9edc-d828ec2b8c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:15.220630 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35545a-a9e4-4a98-9edc-d828ec2b8c09 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:15.221505 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da35545a-a9e4-4a98-9edc-d828ec2b8c09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:15.221969 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da35545a-a9e4-4a98-9edc-d828ec2b8c09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:15.223366 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 295/600] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jun 15 19:25:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:15.224106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da35545a-a9e4-4a98-9edc-d828ec2b8c09 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:15.224429 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e56bb75-98a2-48e8-8dbc-d4b950a2936c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:15.240599 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 306/601] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:15.242308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35545a-a9e4-4a98-9edc-d828ec2b8c09 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:15.303096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:15.327488 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:15.343996 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e56bb75-98a2-48e8-8dbc-d4b950a2936c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:15.381851 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43be0e-279a-400c-b243-908790ea19c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:15.382656 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43be0e-279a-400c-b243-908790ea19c3 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:15.383020 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43be0e-279a-400c-b243-908790ea19c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:15.383643 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43be0e-279a-400c-b243-908790ea19c3 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:15.384443 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba43be0e-279a-400c-b243-908790ea19c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:15.384818 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba43be0e-279a-400c-b243-908790ea19c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:15.386624 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35545a-a9e4-4a98-9edc-d828ec2b8c09 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:15.386945 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba43be0e-279a-400c-b243-908790ea19c3 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:15.405144 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 296/602] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jun 15 19:25:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:15.405144 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43be0e-279a-400c-b243-908790ea19c3 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:15.427635 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7b837-db97-4cc0-81b9-005a3141d7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:15.428522 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7b837-db97-4cc0-81b9-005a3141d7f9 None admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:15.429031 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7b837-db97-4cc0-81b9-005a3141d7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:15.429140 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7b837-db97-4cc0-81b9-005a3141d7f9 None admin] PATH_INFO: `/v3/projects/7fb8cc0eb560465e972ffd45867ee73b/users/8bfca4d302aa43249609513f35d55d7a/roles/a8df9714563b435e9fce55a04e74ae5e` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:15.443717 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7b837-db97-4cc0-81b9-005a3141d7f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=7fb8cc0eb560465e972ffd45867ee73b, user_id=8bfca4d302aa43249609513f35d55d7a, role_id=a8df9714563b435e9fce55a04e74ae5e)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:15.445159 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7b837-db97-4cc0-81b9-005a3141d7f9 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:15.458956 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 307/603] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jun 15 19:25:15 2022] PUT /identity/v3/projects/7fb8cc0eb560465e972ffd45867ee73b/users/8bfca4d302aa43249609513f35d55d7a/roles/a8df9714563b435e9fce55a04e74ae5e => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:25:15.460377 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5df56-9619-4715-b0aa-87660e32171b None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:15.460739 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5df56-9619-4715-b0aa-87660e32171b None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:15.461083 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5df56-9619-4715-b0aa-87660e32171b None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:15.466743 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 308/604] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:15.466743 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd5df56-9619-4715-b0aa-87660e32171b None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:15.536039 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43be0e-279a-400c-b243-908790ea19c3 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:15.570138 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:15.576331 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af670c22-93f5-4128-8787-135bac70d338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:15.577581 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af670c22-93f5-4128-8787-135bac70d338 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:15.577959 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af670c22-93f5-4128-8787-135bac70d338 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:15.578350 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af670c22-93f5-4128-8787-135bac70d338 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:15.586377 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af670c22-93f5-4128-8787-135bac70d338 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:15.588194 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af670c22-93f5-4128-8787-135bac70d338 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:15.603639 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd5df56-9619-4715-b0aa-87660e32171b None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:15.605505 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-af670c22-93f5-4128-8787-135bac70d338 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:15.605505 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:15.605505 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:15.605505 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:15.605505 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:15.605505 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:15.605505 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:15.605505 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:15.605505 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:15.605505 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:15.605505 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:15.605505 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:15.605505 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 19:25:15.605505 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 19:25:15.605505 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 19:25:15.606599 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 19:25:15.606599 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:15.606599 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:15.606599 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:15.606599 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:15.606599 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:15.606599 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:15.606599 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:15.606599 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:15.606599 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:15.606599 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:15.606599 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:15.606599 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:15.606599 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:15.606599 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:15.606599 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:15.606599 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:15.607700 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 19:25:15.607700 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 19:25:15.607700 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:25:15.607700 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:25:15.607700 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:25:15.607700 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:25:15.607700 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:15.607700 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:15.609207 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 297/605] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:25:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:15.611031 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5766674-796e-442d-bd6c-af0055cce767 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:15.611157 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5766674-796e-442d-bd6c-af0055cce767 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:15.611493 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5766674-796e-442d-bd6c-af0055cce767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:15.622490 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:15.630440 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-f5766674-796e-442d-bd6c-af0055cce767 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:15.631731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f5766674-796e-442d-bd6c-af0055cce767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:15.652844 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f904fa-40f1-4ae3-bd0d-aa71c6a5b59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:15.653852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f904fa-40f1-4ae3-bd0d-aa71c6a5b59e None admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:15.654166 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f904fa-40f1-4ae3-bd0d-aa71c6a5b59e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:15.654445 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f904fa-40f1-4ae3-bd0d-aa71c6a5b59e None admin] PATH_INFO: `/v3/projects/7fb8cc0eb560465e972ffd45867ee73b/users/ec5882def40a4e0d8a792eecc12440a5/roles/a8df9714563b435e9fce55a04e74ae5e` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:15.672617 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f904fa-40f1-4ae3-bd0d-aa71c6a5b59e None admin] RBAC: Authorizing `identity:create_grant(project_id=7fb8cc0eb560465e972ffd45867ee73b, user_id=ec5882def40a4e0d8a792eecc12440a5, role_id=a8df9714563b435e9fce55a04e74ae5e)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:15.674210 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f904fa-40f1-4ae3-bd0d-aa71c6a5b59e None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:15.691476 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 309/606] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jun 15 19:25:15 2022] PUT /identity/v3/projects/7fb8cc0eb560465e972ffd45867ee73b/users/ec5882def40a4e0d8a792eecc12440a5/roles/a8df9714563b435e9fce55a04e74ae5e => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:25:15.692416 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f904fa-40f1-4ae3-bd0d-aa71c6a5b59e None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:15.715079 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 298/607] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:25:15.740792 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5766674-796e-442d-bd6c-af0055cce767 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:15.823887 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f904fa-40f1-4ae3-bd0d-aa71c6a5b59e None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:15.870313 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0aa460-d703-4e51-90c9-5bfc858bcc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:15.870996 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0aa460-d703-4e51-90c9-5bfc858bcc80 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:15.871638 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0aa460-d703-4e51-90c9-5bfc858bcc80 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:15.872174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0aa460-d703-4e51-90c9-5bfc858bcc80 None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:15.872980 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0aa460-d703-4e51-90c9-5bfc858bcc80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:15.873299 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0aa460-d703-4e51-90c9-5bfc858bcc80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:15.874077 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5766674-796e-442d-bd6c-af0055cce767 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:15.874769 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0aa460-d703-4e51-90c9-5bfc858bcc80 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:15.891370 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 310/608] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jun 15 19:25:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:15.897749 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0aa460-d703-4e51-90c9-5bfc858bcc80 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:15.924349 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:15.934033 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:15.949901 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0811d-2a27-4d9c-9991-937fe3a19973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:15.950749 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0811d-2a27-4d9c-9991-937fe3a19973 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:15.951037 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0811d-2a27-4d9c-9991-937fe3a19973 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:15.951313 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0811d-2a27-4d9c-9991-937fe3a19973 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:15.974697 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0811d-2a27-4d9c-9991-937fe3a19973 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:25:15.985391 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-c4e0811d-2a27-4d9c-9991-937fe3a19973 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:25:15.985391 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:15.985391 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:15.985391 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:15.985391 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:15.985391 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:15.985391 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:15.985391 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:15.985391 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:15.985391 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:15.985391 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:15.985391 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:15.985391 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:25:15.985391 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:25:15.985391 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:25:15.985391 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:25:15.986373 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:25:15.986373 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:25:15.986373 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:25:15.986373 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:25:15.986373 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:15.986373 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:15.986373 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:15.986373 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:15.986373 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:15.986373 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:15.986373 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:15.986373 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:15.986373 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:15.986373 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:15.986373 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:15.986373 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:15.987335 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:15.987335 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:15.987335 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:15.987335 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:15.987335 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:25:15.987335 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:25:15.987335 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:25:15.987335 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:25:15.987335 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:25:15.987335 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:25:15.987335 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:25:15.987335 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:15.987335 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 299/609] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jun 15 19:25:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:25:15.994540 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0811d-2a27-4d9c-9991-937fe3a19973 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:16.054661 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0aa460-d703-4e51-90c9-5bfc858bcc80 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:16.093633 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40644f5f-f74a-49c3-a9dc-0e571030441c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:16.094471 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40644f5f-f74a-49c3-a9dc-0e571030441c None admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:16.094756 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40644f5f-f74a-49c3-a9dc-0e571030441c None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:16.095379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40644f5f-f74a-49c3-a9dc-0e571030441c None admin] PATH_INFO: `/v3/projects/7fb8cc0eb560465e972ffd45867ee73b/users/5c63e0b6ceed4f7cb2e6f9269eb85795/roles/a8df9714563b435e9fce55a04e74ae5e` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:16.112860 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40644f5f-f74a-49c3-a9dc-0e571030441c None admin] RBAC: Authorizing `identity:create_grant(project_id=7fb8cc0eb560465e972ffd45867ee73b, user_id=5c63e0b6ceed4f7cb2e6f9269eb85795, role_id=a8df9714563b435e9fce55a04e74ae5e)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:16.114395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40644f5f-f74a-49c3-a9dc-0e571030441c None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:16.132201 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 311/610] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jun 15 19:25:15 2022] PUT /identity/v3/projects/7fb8cc0eb560465e972ffd45867ee73b/users/5c63e0b6ceed4f7cb2e6f9269eb85795/roles/a8df9714563b435e9fce55a04e74ae5e => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:25:16.172493 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0811d-2a27-4d9c-9991-937fe3a19973 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:16.225285 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dec3b2-4837-4891-a10b-2c60d31d3d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:16.225980 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dec3b2-4837-4891-a10b-2c60d31d3d42 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:16.226287 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dec3b2-4837-4891-a10b-2c60d31d3d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:16.226566 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dec3b2-4837-4891-a10b-2c60d31d3d42 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:16.227336 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dec3b2-4837-4891-a10b-2c60d31d3d42 None admin] RBAC: Adding query filter params (name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:16.227644 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dec3b2-4837-4891-a10b-2c60d31d3d42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:16.229080 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dec3b2-4837-4891-a10b-2c60d31d3d42 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:16.246353 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40644f5f-f74a-49c3-a9dc-0e571030441c None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:16.249150 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 300/611] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jun 15 19:25:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:16.372944 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:16.416270 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40644f5f-f74a-49c3-a9dc-0e571030441c None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:16.462651 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730bf5e-e0c1-4749-b8f7-0d80b73a59f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:16.463998 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730bf5e-e0c1-4749-b8f7-0d80b73a59f1 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:16.464280 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730bf5e-e0c1-4749-b8f7-0d80b73a59f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:16.464554 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730bf5e-e0c1-4749-b8f7-0d80b73a59f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:16.475718 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730bf5e-e0c1-4749-b8f7-0d80b73a59f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:16.477255 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730bf5e-e0c1-4749-b8f7-0d80b73a59f1 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:16.501805 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-2730bf5e-e0c1-4749-b8f7-0d80b73a59f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:16.501805 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:16.501805 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:16.501805 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:16.501805 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:16.501805 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:16.501805 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:16.501805 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:16.501805 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:16.501805 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:16.501805 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:16.501805 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:16.501805 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:16.501805 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:16.501805 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:16.502789 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:16.502789 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:16.502789 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:16.502789 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:16.502789 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:16.502789 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:16.502789 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:16.502789 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:16.502789 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:16.502789 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:16.502789 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:16.502789 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:16.502789 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:16.502789 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:16.502789 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:16.502789 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:16.502789 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:16.503685 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:16.503685 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:16.503685 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:16.503685 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:16.503685 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 312/612] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:16.509416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dec3b2-4837-4891-a10b-2c60d31d3d42 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:16.566156 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:16.623150 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dec3b2-4837-4891-a10b-2c60d31d3d42 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:16.657791 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ad922-c687-43c1-b394-dfe3312de3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:16.658486 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ad922-c687-43c1-b394-dfe3312de3fe None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:16.658771 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ad922-c687-43c1-b394-dfe3312de3fe None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:16.659047 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ad922-c687-43c1-b394-dfe3312de3fe None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:16.659773 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73ad922-c687-43c1-b394-dfe3312de3fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:16.660083 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73ad922-c687-43c1-b394-dfe3312de3fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:16.661451 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73ad922-c687-43c1-b394-dfe3312de3fe None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:16.678239 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 301/613] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:16.679783 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730bf5e-e0c1-4749-b8f7-0d80b73a59f1 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:16.786701 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730bf5e-e0c1-4749-b8f7-0d80b73a59f1 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:16.814310 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:16.828816 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d706f4-e5bb-41d0-bd54-1d8e90e7dfd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:16.829635 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d706f4-e5bb-41d0-bd54-1d8e90e7dfd5 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:16.829920 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d706f4-e5bb-41d0-bd54-1d8e90e7dfd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:16.830229 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d706f4-e5bb-41d0-bd54-1d8e90e7dfd5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:16.840432 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d706f4-e5bb-41d0-bd54-1d8e90e7dfd5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:16.842254 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d706f4-e5bb-41d0-bd54-1d8e90e7dfd5 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:16.858612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-99d706f4-e5bb-41d0-bd54-1d8e90e7dfd5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 19:25:16.858612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:16.858612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:16.858612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:16.858612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:16.858612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:16.858612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:16.858612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:16.858612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:16.858612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:16.858612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:16.858612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:16.858612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 19:25:16.858612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 19:25:16.858612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 19:25:16.858612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 19:25:16.860842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:16.860842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:16.860842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:25:16.860842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:16.860842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:25:16.860842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:16.860842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:16.860842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:16.860842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:16.860842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:16.860842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:16.860842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:16.860842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:16.860842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:16.860842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:16.860842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:16.860842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:16.860842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:16.863087 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:16.863087 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:16.863087 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 19:25:16.863087 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 19:25:16.863087 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 19:25:16.863087 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 19:25:16.863087 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 19:25:16.863087 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 19:25:16.863087 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 19:25:16.863087 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:16.863087 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 313/614] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jun 15 19:25:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:16.866413 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ad922-c687-43c1-b394-dfe3312de3fe None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:16.884852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd19f43-b97c-4dd0-a5f2-b579d85d46d3 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:16.884852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd19f43-b97c-4dd0-a5f2-b579d85d46d3 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:16.884852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd19f43-b97c-4dd0-a5f2-b579d85d46d3 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:16.886237 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 314/615] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:16.891369 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e09b76-72c0-42f2-81cd-1b23193766df None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:16.891897 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e09b76-72c0-42f2-81cd-1b23193766df None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:16.892377 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e09b76-72c0-42f2-81cd-1b23193766df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:16.910048 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-d0e09b76-72c0-42f2-81cd-1b23193766df None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:16.911594 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-d0e09b76-72c0-42f2-81cd-1b23193766df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:16.979390 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ad922-c687-43c1-b394-dfe3312de3fe None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:16.982452 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:17.036898 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 315/616] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 19:25:17.037551 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ca193-ca40-4506-8138-e451c981367f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:17.042123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ca193-ca40-4506-8138-e451c981367f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:17.042123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ca193-ca40-4506-8138-e451c981367f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:17.042123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ca193-ca40-4506-8138-e451c981367f None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:17.042123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7ca193-ca40-4506-8138-e451c981367f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:17.042123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7ca193-ca40-4506-8138-e451c981367f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:17.042123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7ca193-ca40-4506-8138-e451c981367f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:17.078031 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 302/617] 10.222.0.15 () {56 vars in 1135 bytes} [Wed Jun 15 19:25:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:17.081702 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e09b76-72c0-42f2-81cd-1b23193766df None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:17.089854 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ca193-ca40-4506-8138-e451c981367f None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:17.216208 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e09b76-72c0-42f2-81cd-1b23193766df None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:17.234520 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:17.254321 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22cedf-24db-48a5-9fdf-1023d52f1e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:17.255567 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22cedf-24db-48a5-9fdf-1023d52f1e19 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:17.256063 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22cedf-24db-48a5-9fdf-1023d52f1e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:17.256535 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22cedf-24db-48a5-9fdf-1023d52f1e19 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:17.269335 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d22cedf-24db-48a5-9fdf-1023d52f1e19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:25:17.278933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-6d22cedf-24db-48a5-9fdf-1023d52f1e19 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:25:17.278933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:17.278933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:17.278933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:17.278933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:17.278933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:17.278933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:17.278933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:17.278933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:17.278933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:17.278933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:17.278933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:17.278933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:25:17.278933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:25:17.278933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:25:17.278933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:25:17.279878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:25:17.279878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:25:17.279878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:25:17.279878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:25:17.279878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:17.279878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:17.279878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:17.279878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:17.279878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:17.279878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:17.279878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:17.279878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:17.279878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:17.279878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:17.279878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:17.279878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:17.280716 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:17.280716 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:17.280716 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:17.280716 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:17.280716 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:25:17.280716 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:25:17.280716 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:25:17.280716 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:25:17.280716 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:25:17.280716 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:25:17.280716 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:25:17.280716 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:17.283594 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 316/618] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jun 15 19:25:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:17.293396 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22cedf-24db-48a5-9fdf-1023d52f1e19 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:17.295997 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ca193-ca40-4506-8138-e451c981367f None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:17.387911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6464a19-ce01-48ef-828e-e842b9b95f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:17.388633 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6464a19-ce01-48ef-828e-e842b9b95f18 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:17.388918 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6464a19-ce01-48ef-828e-e842b9b95f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:17.389188 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6464a19-ce01-48ef-828e-e842b9b95f18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:17.399762 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6464a19-ce01-48ef-828e-e842b9b95f18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:17.401283 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6464a19-ce01-48ef-828e-e842b9b95f18 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:17.422265 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-f6464a19-ce01-48ef-828e-e842b9b95f18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:17.422265 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:17.422265 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:17.422265 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:17.422265 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:17.422265 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:17.422265 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:17.422265 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:17.422265 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:17.422265 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:17.422265 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:17.422265 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:17.422265 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:17.422265 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:17.422265 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:17.423174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:17.423174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:17.423174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:17.423174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:17.423174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:17.423174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:17.423174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:17.423174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:17.423174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:17.423174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:17.423174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:17.423174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:17.423174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:17.423174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:17.423174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:17.423174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:17.423174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:17.424220 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:17.424220 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:17.424220 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:17.424220 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:17.424220 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 303/619] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:25:17.428996 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6464a19-ce01-48ef-828e-e842b9b95f18 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:17.437155 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22cedf-24db-48a5-9fdf-1023d52f1e19 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:17.458490 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:17.490521 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a1bdc4-e30d-43c0-b433-9f7b1f9bb29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:17.491519 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a1bdc4-e30d-43c0-b433-9f7b1f9bb29a None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:17.492016 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a1bdc4-e30d-43c0-b433-9f7b1f9bb29a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:17.492515 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a1bdc4-e30d-43c0-b433-9f7b1f9bb29a None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:17.493547 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a1bdc4-e30d-43c0-b433-9f7b1f9bb29a None admin] RBAC: Adding query filter params (name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:17.494056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a1bdc4-e30d-43c0-b433-9f7b1f9bb29a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:17.497765 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a1bdc4-e30d-43c0-b433-9f7b1f9bb29a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:17.513735 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 317/620] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jun 15 19:25:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:25:17.518813 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a1bdc4-e30d-43c0-b433-9f7b1f9bb29a None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:17.602469 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:17.624992 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6464a19-ce01-48ef-828e-e842b9b95f18 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:17.704940 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccffd57a-b8fa-4b24-82d9-fba162ec8ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:17.705618 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccffd57a-b8fa-4b24-82d9-fba162ec8ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:17.705903 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccffd57a-b8fa-4b24-82d9-fba162ec8ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:17.706209 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccffd57a-b8fa-4b24-82d9-fba162ec8ae2 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:17.706950 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccffd57a-b8fa-4b24-82d9-fba162ec8ae2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:17.707259 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccffd57a-b8fa-4b24-82d9-fba162ec8ae2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:17.708654 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccffd57a-b8fa-4b24-82d9-fba162ec8ae2 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:17.726866 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a1bdc4-e30d-43c0-b433-9f7b1f9bb29a None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:17.733181 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 304/621] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:25:17.738734 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccffd57a-b8fa-4b24-82d9-fba162ec8ae2 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:17.814978 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01934136-af1e-4f16-a308-4ef52bd0f279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:17.815705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01934136-af1e-4f16-a308-4ef52bd0f279 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:17.815984 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01934136-af1e-4f16-a308-4ef52bd0f279 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:17.816255 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01934136-af1e-4f16-a308-4ef52bd0f279 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:17.834544 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:17.858669 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccffd57a-b8fa-4b24-82d9-fba162ec8ae2 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:17.863204 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01934136-af1e-4f16-a308-4ef52bd0f279 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:17.864921 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01934136-af1e-4f16-a308-4ef52bd0f279 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:17.895760 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9818dec-8a14-42fd-914d-4367f553e6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:17.896613 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9818dec-8a14-42fd-914d-4367f553e6d7 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:17.896901 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9818dec-8a14-42fd-914d-4367f553e6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:17.897175 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9818dec-8a14-42fd-914d-4367f553e6d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:17.904148 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9818dec-8a14-42fd-914d-4367f553e6d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:17.905608 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9818dec-8a14-42fd-914d-4367f553e6d7 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:17.910271 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-01934136-af1e-4f16-a308-4ef52bd0f279 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:17.910271 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:17.910271 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:17.910271 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:17.910271 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:17.910271 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:17.910271 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:17.910271 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:17.910271 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:17.910271 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:17.910271 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:17.910271 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:17.910271 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:17.910271 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:17.910271 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:17.911188 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:17.911188 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:17.911188 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:17.911188 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:17.911188 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:17.911188 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:17.911188 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:17.911188 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:17.911188 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:17.911188 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:17.911188 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:17.911188 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:17.911188 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:17.911188 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:17.911188 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:17.911188 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:17.911188 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:17.912154 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:17.912154 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:17.912154 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:17.912154 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:17.918535 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 318/622] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:25:17.923110 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-b9818dec-8a14-42fd-914d-4367f553e6d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:17.923110 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:17.923110 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:17.923110 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:17.923110 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:17.923110 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:17.923110 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:17.923110 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:17.923110 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:17.923110 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:17.923110 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:17.923110 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:17.923110 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 19:25:17.923110 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 19:25:17.923110 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 19:25:17.923992 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 19:25:17.923992 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:17.923992 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:17.923992 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:17.923992 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:17.923992 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:17.923992 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:17.923992 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:17.923992 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:17.923992 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:17.923992 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:17.923992 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:17.923992 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:17.923992 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:17.923992 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:17.923992 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:17.923992 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:17.924882 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 19:25:17.924882 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 19:25:17.924882 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:25:17.924882 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:25:17.924882 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:25:17.924882 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:25:17.924882 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:17.924882 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:17.924882 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 305/623] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:25:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:25:17.927090 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01934136-af1e-4f16-a308-4ef52bd0f279 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:17.930976 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9818dec-8a14-42fd-914d-4367f553e6d7 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:18.050930 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:18.072290 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01934136-af1e-4f16-a308-4ef52bd0f279 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:18.153836 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9818dec-8a14-42fd-914d-4367f553e6d7 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:18.157316 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf6409-fc4e-45b9-8f7c-848765c0a2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:18.157935 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf6409-fc4e-45b9-8f7c-848765c0a2b9 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:18.158237 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf6409-fc4e-45b9-8f7c-848765c0a2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:18.158509 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf6409-fc4e-45b9-8f7c-848765c0a2b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:18.159313 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaf6409-fc4e-45b9-8f7c-848765c0a2b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:18.159618 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaf6409-fc4e-45b9-8f7c-848765c0a2b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:18.161237 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaf6409-fc4e-45b9-8f7c-848765c0a2b9 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:18.174652 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 319/624] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:18.176107 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc8139-90a0-409e-a89e-bf3770ba9edd None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:18.176405 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc8139-90a0-409e-a89e-bf3770ba9edd None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:18.176691 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc8139-90a0-409e-a89e-bf3770ba9edd None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:18.181711 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 320/625] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:18.181711 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc8139-90a0-409e-a89e-bf3770ba9edd None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:18.204781 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e2383-0339-4575-bc40-e36185d423b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:18.205494 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e2383-0339-4575-bc40-e36185d423b1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:18.205826 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e2383-0339-4575-bc40-e36185d423b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:18.206126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e2383-0339-4575-bc40-e36185d423b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:18.207074 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e2383-0339-4575-bc40-e36185d423b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:18.207442 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e2383-0339-4575-bc40-e36185d423b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:18.208878 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e2383-0339-4575-bc40-e36185d423b1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:18.210762 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:18.222832 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 306/626] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jun 15 19:25:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:18.224092 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9a464-63f0-4308-a7c0-5954d420c629 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:18.224439 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9a464-63f0-4308-a7c0-5954d420c629 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:18.224725 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9a464-63f0-4308-a7c0-5954d420c629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:18.250523 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-20e9a464-63f0-4308-a7c0-5954d420c629 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:18.251724 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-20e9a464-63f0-4308-a7c0-5954d420c629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:18.310038 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc8139-90a0-409e-a89e-bf3770ba9edd None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:18.346173 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 307/627] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:25:18.347283 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9a464-63f0-4308-a7c0-5954d420c629 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:18.359454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea61f90-9b7d-46ab-a0be-c8abc2f3b556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:18.360558 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea61f90-9b7d-46ab-a0be-c8abc2f3b556 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:18.361072 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea61f90-9b7d-46ab-a0be-c8abc2f3b556 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:18.361551 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea61f90-9b7d-46ab-a0be-c8abc2f3b556 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:18.370315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea61f90-9b7d-46ab-a0be-c8abc2f3b556 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:18.372302 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea61f90-9b7d-46ab-a0be-c8abc2f3b556 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:18.397186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-5ea61f90-9b7d-46ab-a0be-c8abc2f3b556 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 19:25:18.397186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:18.397186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:18.397186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:18.397186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:18.397186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:18.397186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:18.397186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:18.397186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:18.397186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:18.397186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:18.397186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:18.397186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 19:25:18.397186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 19:25:18.397186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 19:25:18.397186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 19:25:18.398264 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:18.398264 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:18.398264 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:25:18.398264 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:18.398264 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:25:18.398264 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:18.398264 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:18.398264 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:18.398264 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:18.398264 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:18.398264 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:18.398264 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:18.398264 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:18.398264 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:18.398264 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:18.398264 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:18.398264 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:18.398264 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:18.399213 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:18.399213 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:18.399213 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 19:25:18.399213 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 19:25:18.399213 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 19:25:18.399213 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 19:25:18.399213 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 19:25:18.399213 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 19:25:18.399213 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 19:25:18.399213 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:18.402063 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 321/628] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Jun 15 19:25:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:18.407703 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea61f90-9b7d-46ab-a0be-c8abc2f3b556 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:18.418740 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:18.485672 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9a464-63f0-4308-a7c0-5954d420c629 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:18.510058 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:18.522418 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf90234-8179-44ec-bc20-c90ed324703d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:18.523381 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf90234-8179-44ec-bc20-c90ed324703d None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:18.523681 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf90234-8179-44ec-bc20-c90ed324703d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:18.523859 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf90234-8179-44ec-bc20-c90ed324703d None admin] PATH_INFO: `/v3/projects/7fb8cc0eb560465e972ffd45867ee73b/users/176d5775a70f4524a9b7fe1ce9d426ed/roles/a8df9714563b435e9fce55a04e74ae5e` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:18.526241 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea61f90-9b7d-46ab-a0be-c8abc2f3b556 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:18.540495 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf90234-8179-44ec-bc20-c90ed324703d None admin] RBAC: Authorizing `identity:create_grant(project_id=7fb8cc0eb560465e972ffd45867ee73b, user_id=176d5775a70f4524a9b7fe1ce9d426ed, role_id=a8df9714563b435e9fce55a04e74ae5e)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:18.542586 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf90234-8179-44ec-bc20-c90ed324703d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:18.556406 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 308/629] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jun 15 19:25:18 2022] PUT /identity/v3/projects/7fb8cc0eb560465e972ffd45867ee73b/users/176d5775a70f4524a9b7fe1ce9d426ed/roles/a8df9714563b435e9fce55a04e74ae5e => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:25:18.557546 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf90234-8179-44ec-bc20-c90ed324703d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:18.574488 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e0c39a-216c-4d10-932a-a6e27f105daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:18.575621 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0c39a-216c-4d10-932a-a6e27f105daf None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:18.575980 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0c39a-216c-4d10-932a-a6e27f105daf None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:18.576471 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0c39a-216c-4d10-932a-a6e27f105daf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:18.590587 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e0c39a-216c-4d10-932a-a6e27f105daf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:25:18.599719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-f6e0c39a-216c-4d10-932a-a6e27f105daf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:25:18.599719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:18.599719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:18.599719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:18.599719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:18.599719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:18.599719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:18.599719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:18.599719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:18.599719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:18.599719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:18.599719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:18.599719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:25:18.599719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:25:18.599719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:25:18.599719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:25:18.600735 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:25:18.600735 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:25:18.600735 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:25:18.600735 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:25:18.600735 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:18.600735 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:18.600735 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:18.600735 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:18.600735 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:18.600735 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:18.600735 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:18.600735 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:18.600735 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:18.600735 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:18.600735 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:18.600735 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:18.601619 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:18.601619 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:18.601619 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:18.601619 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:18.601619 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:25:18.601619 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:25:18.601619 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:25:18.601619 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:25:18.601619 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:25:18.601619 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:25:18.601619 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:25:18.601619 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:18.601619 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 322/630] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jun 15 19:25:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:18.602922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73fa5e-b569-4598-bc97-275403ce25c5 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:18.603424 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73fa5e-b569-4598-bc97-275403ce25c5 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:18.603778 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73fa5e-b569-4598-bc97-275403ce25c5 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:18.606133 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 323/631] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:18.607909 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e73fa5e-b569-4598-bc97-275403ce25c5 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:18.762122 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:18.765647 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf90234-8179-44ec-bc20-c90ed324703d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:18.837217 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210bc1e-8e55-48b0-af46-646a0a19d899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:18.838509 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210bc1e-8e55-48b0-af46-646a0a19d899 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:18.839028 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210bc1e-8e55-48b0-af46-646a0a19d899 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:18.839350 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e73fa5e-b569-4598-bc97-275403ce25c5 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:18.839471 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210bc1e-8e55-48b0-af46-646a0a19d899 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:18.842133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3210bc1e-8e55-48b0-af46-646a0a19d899 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:18.842133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3210bc1e-8e55-48b0-af46-646a0a19d899 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:18.843726 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3210bc1e-8e55-48b0-af46-646a0a19d899 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:18.869072 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 309/632] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Jun 15 19:25:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:18.877790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c969610-04a2-4523-9fc7-462643bf562d None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:18.877790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c969610-04a2-4523-9fc7-462643bf562d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:18.877790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c969610-04a2-4523-9fc7-462643bf562d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:18.902194 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-4c969610-04a2-4523-9fc7-462643bf562d None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:18.902194 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-4c969610-04a2-4523-9fc7-462643bf562d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:18.906109 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7126ba67-7b4b-4314-9da9-ac0f5f52d059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:18.906109 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126ba67-7b4b-4314-9da9-ac0f5f52d059 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:18.906109 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126ba67-7b4b-4314-9da9-ac0f5f52d059 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:18.906109 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126ba67-7b4b-4314-9da9-ac0f5f52d059 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:18.906505 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7126ba67-7b4b-4314-9da9-ac0f5f52d059 None admin] RBAC: Adding query filter params (name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:18.906505 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7126ba67-7b4b-4314-9da9-ac0f5f52d059 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:18.906939 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7126ba67-7b4b-4314-9da9-ac0f5f52d059 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:18.923651 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 324/633] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jun 15 19:25:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:18.925473 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ca0a5-8adf-4601-b176-a9005f8bc785 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:18.926371 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ca0a5-8adf-4601-b176-a9005f8bc785 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:18.926948 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ca0a5-8adf-4601-b176-a9005f8bc785 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:18.930515 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 325/634] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:18.932627 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ca0a5-8adf-4601-b176-a9005f8bc785 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:18.950817 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:19.025010 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 310/635] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:25:19.031088 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c969610-04a2-4523-9fc7-462643bf562d None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:19.104234 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ca0a5-8adf-4601-b176-a9005f8bc785 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:19.144829 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:19.155472 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c969610-04a2-4523-9fc7-462643bf562d None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:19.156708 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d3aabc-8339-47a4-9578-4f799cc6cac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:19.157516 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3aabc-8339-47a4-9578-4f799cc6cac8 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:19.158001 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3aabc-8339-47a4-9578-4f799cc6cac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:19.158363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3aabc-8339-47a4-9578-4f799cc6cac8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:19.194957 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6534bb73-c5b9-4b95-a545-ac72a94b4d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:19.195753 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534bb73-c5b9-4b95-a545-ac72a94b4d36 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:19.196033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534bb73-c5b9-4b95-a545-ac72a94b4d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:19.196446 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534bb73-c5b9-4b95-a545-ac72a94b4d36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:19.207278 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6534bb73-c5b9-4b95-a545-ac72a94b4d36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:19.208725 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6534bb73-c5b9-4b95-a545-ac72a94b4d36 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:19.226351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-6534bb73-c5b9-4b95-a545-ac72a94b4d36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:19.226351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:19.226351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:19.226351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:19.226351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:19.226351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:19.226351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:19.226351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:19.226351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:19.226351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:19.226351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:19.226351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:19.226351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:19.226351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:19.226351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:19.227283 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:19.227283 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:19.227283 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:19.227283 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:19.227283 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:19.227283 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:19.227283 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:19.227283 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:19.227283 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:19.227283 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:19.227283 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:19.227283 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:19.227283 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:19.227283 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:19.227283 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:19.227283 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:19.227283 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:19.228243 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:19.228243 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:19.228243 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:19.228243 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:19.228243 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 311/636] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:19.229540 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ffbcb-37d2-4603-88c1-ae2d95003159 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:19.229819 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ffbcb-37d2-4603-88c1-ae2d95003159 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:19.230148 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ffbcb-37d2-4603-88c1-ae2d95003159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:19.241897 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d3aabc-8339-47a4-9578-4f799cc6cac8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:19.243382 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d3aabc-8339-47a4-9578-4f799cc6cac8 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:19.247478 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-3b1ffbcb-37d2-4603-88c1-ae2d95003159 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:19.248647 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3b1ffbcb-37d2-4603-88c1-ae2d95003159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:19.257946 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:19.318483 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 312/637] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:25:19.319571 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ffbcb-37d2-4603-88c1-ae2d95003159 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:19.359691 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 326/638] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:25:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 19:25:19.360585 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d3aabc-8339-47a4-9578-4f799cc6cac8 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:19.387354 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:19.441567 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ffbcb-37d2-4603-88c1-ae2d95003159 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:19.464825 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d3aabc-8339-47a4-9578-4f799cc6cac8 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:19.475955 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d3ec19-9b00-4e9f-a2ef-cfbcd328b747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:19.476746 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3ec19-9b00-4e9f-a2ef-cfbcd328b747 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:19.477085 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3ec19-9b00-4e9f-a2ef-cfbcd328b747 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:19.477359 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3ec19-9b00-4e9f-a2ef-cfbcd328b747 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:19.487714 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d3ec19-9b00-4e9f-a2ef-cfbcd328b747 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:25:19.495988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-b1d3ec19-9b00-4e9f-a2ef-cfbcd328b747 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:25:19.495988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:19.495988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:19.495988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:19.495988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:19.495988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:19.495988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:19.495988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:19.495988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:19.495988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:19.495988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:19.495988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:19.495988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:25:19.495988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:25:19.495988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:25:19.495988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:25:19.496934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:25:19.496934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:25:19.496934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:25:19.496934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:25:19.496934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:19.496934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:19.496934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:19.496934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:19.496934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:19.496934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:19.496934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:19.496934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:19.496934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:19.496934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:19.496934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:19.496934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:19.497790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:19.497790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:19.497790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:19.497790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:19.497790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:25:19.497790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:25:19.497790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:25:19.497790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:25:19.497790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:25:19.497790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:25:19.497790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:25:19.497790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:19.501866 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 313/639] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jun 15 19:25:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:19.502631 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d3ec19-9b00-4e9f-a2ef-cfbcd328b747 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:19.503209 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dacaf6-7a9a-4489-9edb-80153748dea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:19.503867 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dacaf6-7a9a-4489-9edb-80153748dea9 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:19.504147 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dacaf6-7a9a-4489-9edb-80153748dea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:19.504418 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dacaf6-7a9a-4489-9edb-80153748dea9 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:19.505129 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dacaf6-7a9a-4489-9edb-80153748dea9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:19.505428 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dacaf6-7a9a-4489-9edb-80153748dea9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:19.506865 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dacaf6-7a9a-4489-9edb-80153748dea9 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:19.507297 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:19.519258 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 327/640] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:19.520059 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dacaf6-7a9a-4489-9edb-80153748dea9 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:19.613876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:19.617069 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d3ec19-9b00-4e9f-a2ef-cfbcd328b747 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:19.627307 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dacaf6-7a9a-4489-9edb-80153748dea9 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:19.661977 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcca1f5-2182-4df4-b1c2-6b431f3b3c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:19.662782 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcca1f5-2182-4df4-b1c2-6b431f3b3c77 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:19.663063 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcca1f5-2182-4df4-b1c2-6b431f3b3c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:19.663336 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcca1f5-2182-4df4-b1c2-6b431f3b3c77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:19.666671 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6725120c-2446-4bb5-a1e7-872e9d1146c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:19.667782 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725120c-2446-4bb5-a1e7-872e9d1146c0 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:19.668177 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725120c-2446-4bb5-a1e7-872e9d1146c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:19.668560 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725120c-2446-4bb5-a1e7-872e9d1146c0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:19.673945 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcca1f5-2182-4df4-b1c2-6b431f3b3c77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:19.675400 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcca1f5-2182-4df4-b1c2-6b431f3b3c77 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:19.684798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6725120c-2446-4bb5-a1e7-872e9d1146c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:25:19.692457 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-3fcca1f5-2182-4df4-b1c2-6b431f3b3c77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:19.692457 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:19.692457 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:19.692457 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:19.692457 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:19.692457 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:19.692457 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:19.692457 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:19.692457 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:19.692457 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:19.692457 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:19.692457 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:19.692457 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:19.692457 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:19.692457 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:19.693319 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:19.693319 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:19.693319 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:19.693319 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:19.693319 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:19.693319 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:19.693319 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:19.693319 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:19.693319 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:19.693319 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:19.693319 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:19.693319 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:19.693319 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:19.693319 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:19.693319 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:19.693319 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:19.693319 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:19.694606 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:19.694606 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:19.694606 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:19.694606 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:19.694606 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 328/641] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:19.695223 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcca1f5-2182-4df4-b1c2-6b431f3b3c77 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:19.701657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-6725120c-2446-4bb5-a1e7-872e9d1146c0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:25:19.701657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:19.701657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:19.701657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:19.701657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:19.701657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:19.701657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:19.701657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:19.701657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:19.701657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:19.701657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:19.701657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:19.701657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:25:19.701657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:25:19.701657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:25:19.701657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:25:19.702694 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:25:19.702694 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:25:19.702694 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:25:19.702694 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:25:19.702694 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:19.702694 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:19.702694 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:19.702694 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:19.702694 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:19.702694 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:19.702694 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:19.702694 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:19.702694 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:19.702694 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:19.702694 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:19.702694 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:19.703550 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:19.703550 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:19.703550 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:19.703550 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:19.703550 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:25:19.703550 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:25:19.703550 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:25:19.703550 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:25:19.703550 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:25:19.703550 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:25:19.703550 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:25:19.703550 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:19.706251 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 314/642] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jun 15 19:25:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:19.707150 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6725120c-2446-4bb5-a1e7-872e9d1146c0 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:19.771697 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:19.806159 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcca1f5-2182-4df4-b1c2-6b431f3b3c77 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:19.813985 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6725120c-2446-4bb5-a1e7-872e9d1146c0 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:19.842513 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36127fd-aaaf-4414-ad3a-0fe82b088081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:19.843432 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36127fd-aaaf-4414-ad3a-0fe82b088081 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:19.843886 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36127fd-aaaf-4414-ad3a-0fe82b088081 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:19.844165 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36127fd-aaaf-4414-ad3a-0fe82b088081 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:19.844914 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36127fd-aaaf-4414-ad3a-0fe82b088081 None admin] RBAC: Adding query filter params (name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:19.845221 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36127fd-aaaf-4414-ad3a-0fe82b088081 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:19.846611 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36127fd-aaaf-4414-ad3a-0fe82b088081 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:19.848582 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9a5e1-1cca-451e-b33a-560dba3767ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:19.849705 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9a5e1-1cca-451e-b33a-560dba3767ef None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:19.849989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9a5e1-1cca-451e-b33a-560dba3767ef None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:19.850291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9a5e1-1cca-451e-b33a-560dba3767ef None admin] PATH_INFO: `/v3/users/glance` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:19.859639 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 329/643] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jun 15 19:25:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:19.859639 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36127fd-aaaf-4414-ad3a-0fe82b088081 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:19.861352 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa9a5e1-1cca-451e-b33a-560dba3767ef None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:19.862350 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:19.862841 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa9a5e1-1cca-451e-b33a-560dba3767ef None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:19.877673 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-5fa9a5e1-1cca-451e-b33a-560dba3767ef None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 15 19:25:19.877673 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:19.877673 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:19.877673 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:19.877673 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:19.877673 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:19.877673 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:19.877673 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:19.877673 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:19.877673 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:19.877673 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:19.877673 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:19.877673 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 19:25:19.877673 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 19:25:19.877673 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 19:25:19.877673 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 19:25:19.878822 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:19.878822 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:19.878822 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:25:19.878822 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:19.878822 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:25:19.878822 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:19.878822 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:19.878822 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:19.878822 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:19.878822 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:19.878822 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:19.878822 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:19.878822 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:19.878822 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:19.878822 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:19.878822 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:19.878822 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:19.878822 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:19.880027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:19.880027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:19.880027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 19:25:19.880027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 19:25:19.880027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 19:25:19.880027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 19:25:19.880027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 19:25:19.880027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 19:25:19.880027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 15 19:25:19.880027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:19.880027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 315/644] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:25:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:19.880027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9a5e1-1cca-451e-b33a-560dba3767ef None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:19.975796 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36127fd-aaaf-4414-ad3a-0fe82b088081 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:19.985290 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9a5e1-1cca-451e-b33a-560dba3767ef None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:20.009952 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:20.010756 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcca302-7b82-449f-9111-6eb3f8434f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:20.011413 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcca302-7b82-449f-9111-6eb3f8434f21 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:20.011711 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcca302-7b82-449f-9111-6eb3f8434f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:20.012016 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcca302-7b82-449f-9111-6eb3f8434f21 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:20.012738 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcca302-7b82-449f-9111-6eb3f8434f21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:20.012995 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcca302-7b82-449f-9111-6eb3f8434f21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:20.014379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcca302-7b82-449f-9111-6eb3f8434f21 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:20.023705 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceab05d4-8329-4423-bab0-744942234177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:20.024572 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab05d4-8329-4423-bab0-744942234177 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:20.025029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab05d4-8329-4423-bab0-744942234177 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:20.025494 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab05d4-8329-4423-bab0-744942234177 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:20.026763 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 330/645] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:20.027087 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceab05d4-8329-4423-bab0-744942234177 None admin] RBAC: Adding query filter params (name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:20.027517 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceab05d4-8329-4423-bab0-744942234177 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:20.027614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcca302-7b82-449f-9111-6eb3f8434f21 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:20.029142 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceab05d4-8329-4423-bab0-744942234177 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:20.039395 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 316/646] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jun 15 19:25:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:20.040499 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceab05d4-8329-4423-bab0-744942234177 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:20.096157 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:20.137083 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcca302-7b82-449f-9111-6eb3f8434f21 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:20.151596 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceab05d4-8329-4423-bab0-744942234177 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:20.175080 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a5f8ec-a494-49be-97e2-e210894d913b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:20.175886 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5f8ec-a494-49be-97e2-e210894d913b None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:20.176173 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5f8ec-a494-49be-97e2-e210894d913b None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:20.176451 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5f8ec-a494-49be-97e2-e210894d913b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:20.187421 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a5f8ec-a494-49be-97e2-e210894d913b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:20.188303 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fb2774-80eb-4d92-b89f-d9d046cadf22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:20.188944 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a5f8ec-a494-49be-97e2-e210894d913b None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:20.189675 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb2774-80eb-4d92-b89f-d9d046cadf22 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:20.190168 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb2774-80eb-4d92-b89f-d9d046cadf22 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:20.190652 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb2774-80eb-4d92-b89f-d9d046cadf22 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:20.191557 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fb2774-80eb-4d92-b89f-d9d046cadf22 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:20.191865 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fb2774-80eb-4d92-b89f-d9d046cadf22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:20.193271 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fb2774-80eb-4d92-b89f-d9d046cadf22 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:20.207412 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-19a5f8ec-a494-49be-97e2-e210894d913b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:20.207412 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:20.207412 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:20.207412 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:20.207412 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:20.207412 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:20.207412 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:20.207412 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:20.207412 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:20.207412 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:20.207412 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:20.207412 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:20.207412 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:20.207412 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:20.207412 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:20.208326 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:20.208326 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:20.208326 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:20.208326 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:20.208326 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:20.208326 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:20.208326 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:20.208326 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:20.208326 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:20.208326 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:20.208326 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:20.208326 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:20.208326 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:20.208326 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:20.208326 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:20.208326 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:20.208326 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:20.209217 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:20.209217 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:20.209217 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:20.209217 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:20.209217 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 331/647] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:20.209499 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a5f8ec-a494-49be-97e2-e210894d913b None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:20.218368 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 317/648] 10.222.0.15 () {56 vars in 1137 bytes} [Wed Jun 15 19:25:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:20.219983 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fb2774-80eb-4d92-b89f-d9d046cadf22 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:20.267568 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:20.319555 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a5f8ec-a494-49be-97e2-e210894d913b None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:20.326832 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fb2774-80eb-4d92-b89f-d9d046cadf22 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:20.342334 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:20.364940 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8bf361-6b9e-4f5b-927a-12eef16d262b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:20.365157 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2f4e58-2639-4d31-b068-451533853727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:20.365736 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8bf361-6b9e-4f5b-927a-12eef16d262b None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:20.366021 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8bf361-6b9e-4f5b-927a-12eef16d262b None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:20.366131 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f4e58-2639-4d31-b068-451533853727 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:20.366328 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8bf361-6b9e-4f5b-927a-12eef16d262b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:20.366595 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f4e58-2639-4d31-b068-451533853727 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:20.366963 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f4e58-2639-4d31-b068-451533853727 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:20.373911 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8bf361-6b9e-4f5b-927a-12eef16d262b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:20.375475 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8bf361-6b9e-4f5b-927a-12eef16d262b None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:20.378253 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2f4e58-2639-4d31-b068-451533853727 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:20.379786 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2f4e58-2639-4d31-b068-451533853727 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:20.390068 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-2a8bf361-6b9e-4f5b-927a-12eef16d262b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:20.390068 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:20.390068 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:20.390068 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:20.390068 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:20.390068 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:20.390068 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:20.390068 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:20.390068 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:20.390068 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:20.390068 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:20.390068 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:20.390068 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 19:25:20.390068 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 19:25:20.390068 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 19:25:20.391139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 19:25:20.391139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:20.391139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:20.391139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:20.391139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:20.391139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:20.391139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:20.391139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:20.391139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:20.391139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:20.391139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:20.391139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:20.391139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:20.391139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:20.391139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:20.391139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:20.391139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:20.392331 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 19:25:20.392331 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 19:25:20.392331 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:25:20.392331 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:25:20.392331 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:25:20.392331 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:25:20.392331 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:20.392331 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:20.396032 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 332/649] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:25:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:20.396612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8bf361-6b9e-4f5b-927a-12eef16d262b None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:20.400668 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-bc2f4e58-2639-4d31-b068-451533853727 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:20.400668 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:20.400668 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:20.400668 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:20.400668 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:20.400668 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:20.400668 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:20.400668 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:20.400668 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:20.400668 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:20.400668 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:20.400668 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:20.400668 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:20.400668 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:20.400668 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:20.401736 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:20.401736 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:20.401736 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:20.401736 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:20.401736 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:20.401736 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:20.401736 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:20.401736 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:20.401736 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:20.401736 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:20.401736 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:20.401736 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:20.401736 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:20.401736 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:20.401736 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:20.401736 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:20.401736 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:20.402727 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:20.402727 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:20.402727 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:20.402727 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:20.402727 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 318/650] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:20.403019 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2f4e58-2639-4d31-b068-451533853727 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:20.501960 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:20.509095 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2f4e58-2639-4d31-b068-451533853727 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:20.509859 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8bf361-6b9e-4f5b-927a-12eef16d262b None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:20.543838 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3907a794-19bb-4153-87b0-2c0d5401b4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:20.544501 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907a794-19bb-4153-87b0-2c0d5401b4c6 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:20.544851 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907a794-19bb-4153-87b0-2c0d5401b4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:20.545190 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907a794-19bb-4153-87b0-2c0d5401b4c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:20.546841 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0fb132-e327-41f1-b81f-c0a685b7cef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:20.547569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fb132-e327-41f1-b81f-c0a685b7cef8 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:20.548041 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fb132-e327-41f1-b81f-c0a685b7cef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:20.548377 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fb132-e327-41f1-b81f-c0a685b7cef8 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:20.549278 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0fb132-e327-41f1-b81f-c0a685b7cef8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:20.549665 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0fb132-e327-41f1-b81f-c0a685b7cef8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:20.551121 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0fb132-e327-41f1-b81f-c0a685b7cef8 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:20.568125 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 333/651] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:20.568125 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0fb132-e327-41f1-b81f-c0a685b7cef8 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:20.599924 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:20.622997 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3907a794-19bb-4153-87b0-2c0d5401b4c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:20.624473 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3907a794-19bb-4153-87b0-2c0d5401b4c6 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:20.669116 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0fb132-e327-41f1-b81f-c0a685b7cef8 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:20.702719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38b5cb9-de15-4415-9e40-44997344e452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:20.703485 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b5cb9-de15-4415-9e40-44997344e452 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:20.703832 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b5cb9-de15-4415-9e40-44997344e452 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:20.704169 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b5cb9-de15-4415-9e40-44997344e452 None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:20.705031 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38b5cb9-de15-4415-9e40-44997344e452 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:20.705391 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38b5cb9-de15-4415-9e40-44997344e452 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:20.706857 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38b5cb9-de15-4415-9e40-44997344e452 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:20.720345 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 334/652] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jun 15 19:25:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:20.721318 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38b5cb9-de15-4415-9e40-44997344e452 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:20.730133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:20.737792 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 319/653] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:25:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 19:25:20.738700 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3907a794-19bb-4153-87b0-2c0d5401b4c6 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:20.830674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:20.837591 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38b5cb9-de15-4415-9e40-44997344e452 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:20.845995 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3907a794-19bb-4153-87b0-2c0d5401b4c6 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:20.871591 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed241e-c750-43a0-a043-c6ebdd4d7013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:20.872248 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed241e-c750-43a0-a043-c6ebdd4d7013 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:20.872530 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed241e-c750-43a0-a043-c6ebdd4d7013 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:20.872805 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed241e-c750-43a0-a043-c6ebdd4d7013 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:20.873518 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed241e-c750-43a0-a043-c6ebdd4d7013 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:20.873821 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed241e-c750-43a0-a043-c6ebdd4d7013 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:20.875212 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed241e-c750-43a0-a043-c6ebdd4d7013 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:20.878424 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560cb41-fc99-421a-b941-17c3bb079844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:20.879209 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560cb41-fc99-421a-b941-17c3bb079844 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:20.879490 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560cb41-fc99-421a-b941-17c3bb079844 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:20.879763 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560cb41-fc99-421a-b941-17c3bb079844 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:20.886493 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 335/654] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:20.886935 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560cb41-fc99-421a-b941-17c3bb079844 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:20.887281 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed241e-c750-43a0-a043-c6ebdd4d7013 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:20.888429 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560cb41-fc99-421a-b941-17c3bb079844 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:20.903804 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-c560cb41-fc99-421a-b941-17c3bb079844 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 19:25:20.903804 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:20.903804 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:20.903804 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:20.903804 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:20.903804 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:20.903804 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:20.903804 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:20.903804 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:20.903804 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:20.903804 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:20.903804 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:20.903804 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 19:25:20.903804 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 19:25:20.903804 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 19:25:20.903804 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 19:25:20.904774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:20.904774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:20.904774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:25:20.904774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:20.904774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:25:20.904774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:20.904774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:20.904774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:20.904774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:20.904774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:20.904774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:20.904774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:20.904774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:20.904774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:20.904774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:20.904774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:20.904774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:20.904774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:20.905712 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:20.905712 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:20.905712 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 19:25:20.905712 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 19:25:20.905712 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 19:25:20.905712 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 19:25:20.905712 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 19:25:20.905712 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 19:25:20.905712 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 19:25:20.905712 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:20.905712 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 320/655] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:25:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:20.905712 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560cb41-fc99-421a-b941-17c3bb079844 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:20.959614 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:20.988899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed241e-c750-43a0-a043-c6ebdd4d7013 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:21.002356 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560cb41-fc99-421a-b941-17c3bb079844 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:21.021547 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1199e5-e165-44ed-bcae-6a7526d0294e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:21.022223 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1199e5-e165-44ed-bcae-6a7526d0294e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:21.022506 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1199e5-e165-44ed-bcae-6a7526d0294e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:21.022780 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1199e5-e165-44ed-bcae-6a7526d0294e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:21.023601 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1199e5-e165-44ed-bcae-6a7526d0294e None admin] RBAC: Adding query filter params (role.id=a8df9714563b435e9fce55a04e74ae5e, scope.project.id=7fb8cc0eb560465e972ffd45867ee73b, user.id=3e74ac44e5814122a3226ac023f3688a) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:21.023910 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1199e5-e165-44ed-bcae-6a7526d0294e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:21.025328 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1199e5-e165-44ed-bcae-6a7526d0294e None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:21.031608 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 336/656] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:25:20 2022] GET /identity/v3/role_assignments?user.id=3e74ac44e5814122a3226ac023f3688a&scope.project.id=7fb8cc0eb560465e972ffd45867ee73b&role.id=a8df9714563b435e9fce55a04e74ae5e => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:21.032790 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030fd9a4-2ce6-4603-ae4c-e09588d7e369 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:21.033077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030fd9a4-2ce6-4603-ae4c-e09588d7e369 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:21.033355 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030fd9a4-2ce6-4603-ae4c-e09588d7e369 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:21.034768 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 337/657] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:21.035293 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030fd9a4-2ce6-4603-ae4c-e09588d7e369 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:21.036159 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8914f7ff-8961-462c-8af2-d151423de41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:21.036958 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914f7ff-8961-462c-8af2-d151423de41d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:21.037238 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914f7ff-8961-462c-8af2-d151423de41d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:21.037509 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914f7ff-8961-462c-8af2-d151423de41d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:21.048769 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8914f7ff-8961-462c-8af2-d151423de41d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:21.050296 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8914f7ff-8961-462c-8af2-d151423de41d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:21.063734 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:21.071248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-8914f7ff-8961-462c-8af2-d151423de41d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:21.071248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:21.071248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:21.071248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:21.071248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:21.071248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:21.071248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:21.071248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:21.071248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:21.071248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:21.071248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:21.071248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:21.071248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:21.071248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:21.071248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:21.072251 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:21.072251 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:21.072251 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:21.072251 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:21.072251 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:21.072251 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:21.072251 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:21.072251 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:21.072251 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:21.072251 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:21.072251 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:21.072251 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:21.072251 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:21.072251 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:21.072251 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:21.072251 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:21.072251 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:21.073719 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:21.073719 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:21.073719 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:21.073719 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:21.073719 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 321/658] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:21.074013 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8914f7ff-8961-462c-8af2-d151423de41d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:21.165617 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030fd9a4-2ce6-4603-ae4c-e09588d7e369 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:21.188908 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8914f7ff-8961-462c-8af2-d151423de41d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:21.199838 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:21.210077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa71cf-0214-4313-a87c-360826bc5568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:21.210906 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa71cf-0214-4313-a87c-360826bc5568 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:21.211791 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa71cf-0214-4313-a87c-360826bc5568 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:21.211791 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa71cf-0214-4313-a87c-360826bc5568 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:21.220137 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffa71cf-0214-4313-a87c-360826bc5568 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:21.221586 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffa71cf-0214-4313-a87c-360826bc5568 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:21.233898 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327eeea4-e9ca-4a43-9eb3-5e7273e9ec95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:21.234809 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327eeea4-e9ca-4a43-9eb3-5e7273e9ec95 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:21.235245 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327eeea4-e9ca-4a43-9eb3-5e7273e9ec95 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:21.235517 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327eeea4-e9ca-4a43-9eb3-5e7273e9ec95 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:21.236362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327eeea4-e9ca-4a43-9eb3-5e7273e9ec95 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:21.236674 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327eeea4-e9ca-4a43-9eb3-5e7273e9ec95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:21.238195 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327eeea4-e9ca-4a43-9eb3-5e7273e9ec95 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:21.238466 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-5ffa71cf-0214-4313-a87c-360826bc5568 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 19:25:21.238466 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:21.238466 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:21.238466 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:21.238466 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:21.238466 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:21.238466 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:21.238466 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:21.238466 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:21.238466 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:21.238466 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:21.238466 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:21.238466 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 19:25:21.238466 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 19:25:21.238466 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 19:25:21.238466 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 19:25:21.239395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:21.239395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:21.239395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:25:21.239395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:21.239395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:25:21.239395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:21.239395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:21.239395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:21.239395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:21.239395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:21.239395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:21.239395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:21.239395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:21.239395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:21.239395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:21.239395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:21.239395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:21.239395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:21.240828 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:21.240828 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:21.240828 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 19:25:21.240828 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 19:25:21.240828 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 19:25:21.240828 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 19:25:21.240828 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 19:25:21.240828 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 19:25:21.240828 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 19:25:21.240828 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:21.240828 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 338/659] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jun 15 19:25:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:21.241561 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca21c2-46e0-4a09-82be-a2895b5fc190 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:21.241561 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca21c2-46e0-4a09-82be-a2895b5fc190 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:21.241744 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca21c2-46e0-4a09-82be-a2895b5fc190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:21.257487 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 322/660] 10.222.0.15 () {56 vars in 1137 bytes} [Wed Jun 15 19:25:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:21.261240 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327eeea4-e9ca-4a43-9eb3-5e7273e9ec95 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:21.268315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-8aca21c2-46e0-4a09-82be-a2895b5fc190 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:21.269511 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-8aca21c2-46e0-4a09-82be-a2895b5fc190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:21.346375 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 339/661] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:25:21.347818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca21c2-46e0-4a09-82be-a2895b5fc190 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:21.363163 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:21.375417 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327eeea4-e9ca-4a43-9eb3-5e7273e9ec95 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:21.422002 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c34fc6-3137-48ba-92b0-8ac63e92092c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:21.422817 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c34fc6-3137-48ba-92b0-8ac63e92092c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:21.422992 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c34fc6-3137-48ba-92b0-8ac63e92092c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:21.423267 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c34fc6-3137-48ba-92b0-8ac63e92092c None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:21.424000 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c34fc6-3137-48ba-92b0-8ac63e92092c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:21.424305 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c34fc6-3137-48ba-92b0-8ac63e92092c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:21.425690 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c34fc6-3137-48ba-92b0-8ac63e92092c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:21.442769 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 323/662] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:21.442769 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c34fc6-3137-48ba-92b0-8ac63e92092c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:21.465167 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca21c2-46e0-4a09-82be-a2895b5fc190 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:21.472375 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:21.502705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f69fd2-fe46-4646-84b8-e580dac08ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:21.503371 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f69fd2-fe46-4646-84b8-e580dac08ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:21.503651 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f69fd2-fe46-4646-84b8-e580dac08ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:21.503925 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f69fd2-fe46-4646-84b8-e580dac08ff6 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:21.504756 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f69fd2-fe46-4646-84b8-e580dac08ff6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:21.505061 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f69fd2-fe46-4646-84b8-e580dac08ff6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:21.506525 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f69fd2-fe46-4646-84b8-e580dac08ff6 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:21.524037 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 340/663] 10.222.0.15 () {56 vars in 1139 bytes} [Wed Jun 15 19:25:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:21.524856 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f69fd2-fe46-4646-84b8-e580dac08ff6 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:21.555573 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c34fc6-3137-48ba-92b0-8ac63e92092c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:21.593299 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65818e23-0db7-46d5-bc13-841365b0c7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:21.593963 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65818e23-0db7-46d5-bc13-841365b0c7e6 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:21.594276 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65818e23-0db7-46d5-bc13-841365b0c7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:21.594557 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65818e23-0db7-46d5-bc13-841365b0c7e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:21.601946 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:21.639571 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f69fd2-fe46-4646-84b8-e580dac08ff6 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:21.668407 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65818e23-0db7-46d5-bc13-841365b0c7e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:21.669750 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65818e23-0db7-46d5-bc13-841365b0c7e6 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:21.676933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0acafd8-7203-43eb-93be-f840b695a5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:21.678171 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0acafd8-7203-43eb-93be-f840b695a5f7 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:21.678528 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0acafd8-7203-43eb-93be-f840b695a5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:21.678875 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0acafd8-7203-43eb-93be-f840b695a5f7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:21.691426 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0acafd8-7203-43eb-93be-f840b695a5f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:25:21.697110 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:21.700181 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-e0acafd8-7203-43eb-93be-f840b695a5f7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:25:21.700181 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:21.700181 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:21.700181 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:21.700181 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:21.700181 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:21.700181 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:21.700181 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:21.700181 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:21.700181 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:21.700181 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:21.700181 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:21.700181 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:25:21.700181 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:25:21.700181 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:25:21.700181 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:25:21.701323 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:25:21.701323 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:25:21.701323 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:25:21.701323 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:25:21.701323 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:21.701323 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:21.701323 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:21.701323 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:21.701323 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:21.701323 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:21.701323 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:21.701323 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:21.701323 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:21.701323 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:21.701323 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:21.701323 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:21.702389 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:21.702389 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:21.702389 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:21.702389 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:21.702389 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:25:21.702389 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:25:21.702389 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:25:21.702389 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:25:21.702389 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:25:21.702389 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:25:21.702389 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:25:21.702389 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:21.703038 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 341/664] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jun 15 19:25:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:21.703542 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0acafd8-7203-43eb-93be-f840b695a5f7 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:21.784981 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 324/665] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:25:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 19:25:21.784981 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65818e23-0db7-46d5-bc13-841365b0c7e6 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:21.812338 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0acafd8-7203-43eb-93be-f840b695a5f7 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:21.840866 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:21.851209 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770bf85-93cf-40db-9223-7884c83d008f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:21.852010 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770bf85-93cf-40db-9223-7884c83d008f None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:21.852298 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770bf85-93cf-40db-9223-7884c83d008f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:21.852573 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770bf85-93cf-40db-9223-7884c83d008f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:21.860303 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f770bf85-93cf-40db-9223-7884c83d008f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:21.861773 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f770bf85-93cf-40db-9223-7884c83d008f None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:21.877259 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-f770bf85-93cf-40db-9223-7884c83d008f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:21.877259 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:21.877259 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:21.877259 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:21.877259 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:21.877259 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:21.877259 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:21.877259 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:21.877259 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:21.877259 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:21.877259 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:21.877259 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:21.877259 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 19:25:21.877259 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 19:25:21.877259 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 19:25:21.878206 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 19:25:21.878206 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:21.878206 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:21.878206 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:21.878206 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:21.878206 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:21.878206 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:21.878206 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:21.878206 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:21.878206 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:21.878206 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:21.878206 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:21.878206 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:21.878206 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:21.878206 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:21.878206 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:21.878206 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:21.879095 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 19:25:21.879095 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 19:25:21.879095 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:25:21.879095 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:25:21.879095 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:25:21.879095 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:25:21.879095 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:21.879095 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:21.879095 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 342/666] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:25:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:21.879586 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770bf85-93cf-40db-9223-7884c83d008f None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:21.890911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65818e23-0db7-46d5-bc13-841365b0c7e6 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:21.921908 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:21.933185 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd40fca0-7f80-443f-9079-dab9f0c8ec3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:21.933838 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40fca0-7f80-443f-9079-dab9f0c8ec3e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:21.934130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40fca0-7f80-443f-9079-dab9f0c8ec3e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:21.934409 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40fca0-7f80-443f-9079-dab9f0c8ec3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:21.985614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770bf85-93cf-40db-9223-7884c83d008f None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:22.006536 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd40fca0-7f80-443f-9079-dab9f0c8ec3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:22.007990 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd40fca0-7f80-443f-9079-dab9f0c8ec3e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:22.021209 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7180f-2067-484a-891e-b7ff6831cc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:22.021869 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7180f-2067-484a-891e-b7ff6831cc7f None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:22.022233 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7180f-2067-484a-891e-b7ff6831cc7f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:22.022453 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7180f-2067-484a-891e-b7ff6831cc7f None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:22.023226 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b7180f-2067-484a-891e-b7ff6831cc7f None admin] RBAC: Adding query filter params (name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:22.023493 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b7180f-2067-484a-891e-b7ff6831cc7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:22.024836 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b7180f-2067-484a-891e-b7ff6831cc7f None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:22.032924 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 343/667] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jun 15 19:25:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:22.033701 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7180f-2067-484a-891e-b7ff6831cc7f None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:22.086244 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:22.118544 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 325/668] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:25:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 19:25:22.119813 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd40fca0-7f80-443f-9079-dab9f0c8ec3e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:22.147506 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7180f-2067-484a-891e-b7ff6831cc7f None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:22.148396 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:22.182608 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab32f8-5a2f-4835-adf8-4b7a56621bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:22.183463 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab32f8-5a2f-4835-adf8-4b7a56621bbe None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:22.183690 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab32f8-5a2f-4835-adf8-4b7a56621bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:22.183964 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab32f8-5a2f-4835-adf8-4b7a56621bbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:22.194803 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab32f8-5a2f-4835-adf8-4b7a56621bbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:22.196239 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab32f8-5a2f-4835-adf8-4b7a56621bbe None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:22.213687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-4aab32f8-5a2f-4835-adf8-4b7a56621bbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:22.213687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:22.213687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:22.213687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:22.213687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:22.213687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:22.213687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:22.213687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:22.213687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:22.213687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:22.213687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:22.213687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:22.213687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:22.213687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:22.213687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:22.215098 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:22.215098 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:22.215098 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:22.215098 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:22.215098 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:22.215098 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:22.215098 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:22.215098 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:22.215098 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:22.215098 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:22.215098 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:22.215098 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:22.215098 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:22.215098 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:22.215098 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:22.215098 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:22.215098 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:22.215996 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:22.215996 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:22.215996 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:22.215996 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:22.215996 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 344/669] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:22.215996 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab32f8-5a2f-4835-adf8-4b7a56621bbe None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:22.241237 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd40fca0-7f80-443f-9079-dab9f0c8ec3e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:22.281844 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ada1cb-7a00-4072-bea6-d61d979f8204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:22.282894 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ada1cb-7a00-4072-bea6-d61d979f8204 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:22.283408 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ada1cb-7a00-4072-bea6-d61d979f8204 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:22.283886 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ada1cb-7a00-4072-bea6-d61d979f8204 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:22.285007 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ada1cb-7a00-4072-bea6-d61d979f8204 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:22.285521 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ada1cb-7a00-4072-bea6-d61d979f8204 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:22.291968 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ada1cb-7a00-4072-bea6-d61d979f8204 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:22.320922 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 326/670] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jun 15 19:25:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:22.322116 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ada1cb-7a00-4072-bea6-d61d979f8204 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:22.324507 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:22.327788 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab32f8-5a2f-4835-adf8-4b7a56621bbe None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:22.368784 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce158c76-cbc9-4101-b980-2af98c909e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:22.369616 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce158c76-cbc9-4101-b980-2af98c909e58 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:22.369896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce158c76-cbc9-4101-b980-2af98c909e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:22.370197 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce158c76-cbc9-4101-b980-2af98c909e58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:22.380484 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce158c76-cbc9-4101-b980-2af98c909e58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:22.381917 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce158c76-cbc9-4101-b980-2af98c909e58 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:22.399649 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-ce158c76-cbc9-4101-b980-2af98c909e58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:22.399649 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:22.399649 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:22.399649 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:22.399649 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:22.399649 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:22.399649 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:22.399649 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:22.399649 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:22.399649 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:22.399649 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:22.399649 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:22.399649 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:22.399649 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:22.399649 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:22.401077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:22.401077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:22.401077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:22.401077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:22.401077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:22.401077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:22.401077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:22.401077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:22.401077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:22.401077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:22.401077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:22.401077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:22.401077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:22.401077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:22.401077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:22.401077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:22.401077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:22.401972 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:22.401972 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:22.401972 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:22.401972 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:22.401972 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 345/671] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:22.402274 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce158c76-cbc9-4101-b980-2af98c909e58 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:22.420092 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:22.437569 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ada1cb-7a00-4072-bea6-d61d979f8204 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:22.473470 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b413613d-da1d-4cfc-9c97-60d298d08d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:22.474283 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413613d-da1d-4cfc-9c97-60d298d08d98 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:22.474568 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413613d-da1d-4cfc-9c97-60d298d08d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:22.474840 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413613d-da1d-4cfc-9c97-60d298d08d98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:22.485913 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b413613d-da1d-4cfc-9c97-60d298d08d98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:22.487257 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b413613d-da1d-4cfc-9c97-60d298d08d98 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:22.508671 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-b413613d-da1d-4cfc-9c97-60d298d08d98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:22.508671 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:22.508671 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:22.508671 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:22.508671 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:22.508671 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:22.508671 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:22.508671 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:22.508671 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:22.508671 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:22.508671 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:22.508671 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:22.508671 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:22.508671 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:22.508671 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:22.509708 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:22.509708 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:22.509708 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:22.509708 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:22.509708 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:22.509708 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:22.509708 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:22.509708 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:22.509708 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:22.509708 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:22.509708 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:22.509708 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:22.509708 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:22.509708 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:22.509708 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:22.509708 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:22.509708 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:22.511104 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:22.511104 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:22.511104 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:22.511104 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:22.511104 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 327/672] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:22.511104 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b413613d-da1d-4cfc-9c97-60d298d08d98 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:22.523112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce158c76-cbc9-4101-b980-2af98c909e58 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:22.558885 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1aefe-45b7-46dc-98d0-0d472e26f81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:22.559767 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1aefe-45b7-46dc-98d0-0d472e26f81a None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:22.560195 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1aefe-45b7-46dc-98d0-0d472e26f81a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:22.560639 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1aefe-45b7-46dc-98d0-0d472e26f81a None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:22.561489 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d1aefe-45b7-46dc-98d0-0d472e26f81a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:22.561931 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d1aefe-45b7-46dc-98d0-0d472e26f81a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:22.563468 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d1aefe-45b7-46dc-98d0-0d472e26f81a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:22.576736 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 346/673] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:22.577762 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1aefe-45b7-46dc-98d0-0d472e26f81a None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:22.595314 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:22.622775 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b413613d-da1d-4cfc-9c97-60d298d08d98 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:22.656919 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1084ad3-2dce-4cf0-853e-e9147e395a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:22.657572 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1084ad3-2dce-4cf0-853e-e9147e395a1b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:22.657854 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1084ad3-2dce-4cf0-853e-e9147e395a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:22.658157 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1084ad3-2dce-4cf0-853e-e9147e395a1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:22.658989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1084ad3-2dce-4cf0-853e-e9147e395a1b None admin] RBAC: Adding query filter params (role.id=a8df9714563b435e9fce55a04e74ae5e, scope.project.id=7fb8cc0eb560465e972ffd45867ee73b, user.id=8bfca4d302aa43249609513f35d55d7a) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:22.659301 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1084ad3-2dce-4cf0-853e-e9147e395a1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:22.660726 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1084ad3-2dce-4cf0-853e-e9147e395a1b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:22.670126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 328/674] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:25:22 2022] GET /identity/v3/role_assignments?user.id=8bfca4d302aa43249609513f35d55d7a&scope.project.id=7fb8cc0eb560465e972ffd45867ee73b&role.id=a8df9714563b435e9fce55a04e74ae5e => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:22.670126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1084ad3-2dce-4cf0-853e-e9147e395a1b None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:22.694743 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:22.698886 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1aefe-45b7-46dc-98d0-0d472e26f81a None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:22.752855 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b933bd52-b85b-4d3e-bb1a-23c47a8f4f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:22.753534 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933bd52-b85b-4d3e-bb1a-23c47a8f4f4d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:22.753818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933bd52-b85b-4d3e-bb1a-23c47a8f4f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:22.754116 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933bd52-b85b-4d3e-bb1a-23c47a8f4f4d None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:22.754853 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b933bd52-b85b-4d3e-bb1a-23c47a8f4f4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:22.755160 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b933bd52-b85b-4d3e-bb1a-23c47a8f4f4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:22.756541 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b933bd52-b85b-4d3e-bb1a-23c47a8f4f4d None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:22.769904 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 347/675] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:22.770894 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b933bd52-b85b-4d3e-bb1a-23c47a8f4f4d None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:22.822649 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1084ad3-2dce-4cf0-853e-e9147e395a1b None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:22.847657 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:22.864088 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc02ce13-3ef4-4808-8f52-48814e94558a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:22.864923 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02ce13-3ef4-4808-8f52-48814e94558a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:22.865215 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02ce13-3ef4-4808-8f52-48814e94558a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:22.865500 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02ce13-3ef4-4808-8f52-48814e94558a None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:22.866956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc02ce13-3ef4-4808-8f52-48814e94558a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:22.871000 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc02ce13-3ef4-4808-8f52-48814e94558a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:22.873423 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc02ce13-3ef4-4808-8f52-48814e94558a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:22.887588 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 329/676] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:22.888580 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc02ce13-3ef4-4808-8f52-48814e94558a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:22.889687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b933bd52-b85b-4d3e-bb1a-23c47a8f4f4d None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:22.936782 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aefca4-092d-423c-a8bf-f958b79b3136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:22.938836 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aefca4-092d-423c-a8bf-f958b79b3136 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:22.939138 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aefca4-092d-423c-a8bf-f958b79b3136 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:22.939420 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aefca4-092d-423c-a8bf-f958b79b3136 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:22.950886 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aefca4-092d-423c-a8bf-f958b79b3136 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:22.952321 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aefca4-092d-423c-a8bf-f958b79b3136 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:22.969637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-81aefca4-092d-423c-a8bf-f958b79b3136 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:22.969637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:22.969637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:22.969637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:22.969637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:22.969637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:22.969637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:22.969637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:22.969637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:22.969637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:22.969637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:22.969637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:22.969637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 19:25:22.969637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 19:25:22.969637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 19:25:22.970616 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 19:25:22.970616 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:22.970616 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:22.970616 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:22.970616 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:22.970616 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:22.970616 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:22.970616 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:22.970616 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:22.970616 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:22.970616 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:22.970616 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:22.970616 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:22.970616 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:22.970616 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:22.970616 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:22.970616 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:22.971539 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 19:25:22.971539 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 19:25:22.971539 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:25:22.971539 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:25:22.971539 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:25:22.971539 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:25:22.971539 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:22.971539 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:22.971539 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 348/677] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:25:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:22.972041 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aefca4-092d-423c-a8bf-f958b79b3136 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:22.977005 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:23.025632 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc02ce13-3ef4-4808-8f52-48814e94558a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:23.062304 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798be40e-2c14-41ae-940a-1412fcc6dbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:23.063349 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798be40e-2c14-41ae-940a-1412fcc6dbea None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:23.063783 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798be40e-2c14-41ae-940a-1412fcc6dbea None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:23.064206 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798be40e-2c14-41ae-940a-1412fcc6dbea None admin] PATH_INFO: `/v3/users/swift` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:23.072881 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798be40e-2c14-41ae-940a-1412fcc6dbea None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:23.074938 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798be40e-2c14-41ae-940a-1412fcc6dbea None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:23.093778 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-798be40e-2c14-41ae-940a-1412fcc6dbea None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 19:25:23.093778 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:23.093778 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:23.093778 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:23.093778 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:23.093778 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:23.093778 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:23.093778 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:23.093778 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:23.093778 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:23.093778 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:23.093778 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:23.093778 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 19:25:23.093778 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 19:25:23.093778 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 19:25:23.093778 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 19:25:23.095039 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:23.095039 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:23.095039 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:25:23.095039 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:23.095039 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:25:23.095039 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:23.095039 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:23.095039 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:23.095039 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:23.095039 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:23.095039 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:23.095039 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:23.095039 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:23.095039 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:23.095039 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:23.095039 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:23.095039 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:23.095039 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:23.096258 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:23.096258 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:23.096258 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 19:25:23.096258 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 19:25:23.096258 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 19:25:23.096258 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 19:25:23.096258 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 19:25:23.096258 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 19:25:23.096258 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 19:25:23.096258 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:23.098432 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 330/678] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jun 15 19:25:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:23.099475 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798be40e-2c14-41ae-940a-1412fcc6dbea None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:23.107281 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aefca4-092d-423c-a8bf-f958b79b3136 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:23.149490 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:23.157892 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850d840-4292-4e4e-bdce-68eada97e59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:23.158729 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850d840-4292-4e4e-bdce-68eada97e59e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:23.159018 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850d840-4292-4e4e-bdce-68eada97e59e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:23.159299 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850d840-4292-4e4e-bdce-68eada97e59e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:23.169798 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6850d840-4292-4e4e-bdce-68eada97e59e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:23.172147 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6850d840-4292-4e4e-bdce-68eada97e59e None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:23.190435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-6850d840-4292-4e4e-bdce-68eada97e59e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:23.190435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:23.190435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:23.190435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:23.190435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:23.190435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:23.190435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:23.190435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:23.190435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:23.190435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:23.190435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:23.190435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:23.190435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 19:25:23.190435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 19:25:23.190435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 19:25:23.191406 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 19:25:23.191406 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:23.191406 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:23.191406 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:23.191406 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:23.191406 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:23.191406 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:23.191406 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:23.191406 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:23.191406 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:23.191406 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:23.191406 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:23.191406 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:23.191406 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:23.191406 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:23.191406 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:23.191406 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:23.192291 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 19:25:23.192291 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 19:25:23.192291 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:25:23.192291 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:25:23.192291 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:25:23.192291 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:25:23.192291 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:23.192291 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:23.194963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 349/679] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:25:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:23.196120 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850d840-4292-4e4e-bdce-68eada97e59e None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:23.232492 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798be40e-2c14-41ae-940a-1412fcc6dbea None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:23.265556 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:23.277762 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6f9667-9e68-4ed9-9ecd-b37f51436ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:23.278450 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f9667-9e68-4ed9-9ecd-b37f51436ac2 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:23.278733 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f9667-9e68-4ed9-9ecd-b37f51436ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:23.279009 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f9667-9e68-4ed9-9ecd-b37f51436ac2 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:23.279794 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6f9667-9e68-4ed9-9ecd-b37f51436ac2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:23.280101 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6f9667-9e68-4ed9-9ecd-b37f51436ac2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:23.281536 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6f9667-9e68-4ed9-9ecd-b37f51436ac2 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:23.295130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 331/680] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jun 15 19:25:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:23.295953 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6f9667-9e68-4ed9-9ecd-b37f51436ac2 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:23.324476 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850d840-4292-4e4e-bdce-68eada97e59e None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:23.365837 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f7b6e2-ff19-4d03-8bfa-d4336fd91bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:23.366537 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f7b6e2-ff19-4d03-8bfa-d4336fd91bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:23.366823 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f7b6e2-ff19-4d03-8bfa-d4336fd91bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:23.367097 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f7b6e2-ff19-4d03-8bfa-d4336fd91bb9 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:23.367932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f7b6e2-ff19-4d03-8bfa-d4336fd91bb9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:23.368241 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f7b6e2-ff19-4d03-8bfa-d4336fd91bb9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:23.370523 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f7b6e2-ff19-4d03-8bfa-d4336fd91bb9 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:23.388366 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 350/681] 10.222.0.15 () {56 vars in 1135 bytes} [Wed Jun 15 19:25:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:23.389477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851ba2d-6a63-4a5b-9fc0-4f7a23aea745 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:23.389772 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851ba2d-6a63-4a5b-9fc0-4f7a23aea745 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:23.390051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851ba2d-6a63-4a5b-9fc0-4f7a23aea745 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:23.394201 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 351/682] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:23.394201 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851ba2d-6a63-4a5b-9fc0-4f7a23aea745 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:23.418971 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6f9667-9e68-4ed9-9ecd-b37f51436ac2 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:23.445053 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:23.455457 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b509e7-1ca6-4587-a320-c9172e6c211a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:23.456318 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b509e7-1ca6-4587-a320-c9172e6c211a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:23.456774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b509e7-1ca6-4587-a320-c9172e6c211a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:23.457055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b509e7-1ca6-4587-a320-c9172e6c211a None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:23.457854 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b509e7-1ca6-4587-a320-c9172e6c211a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:23.458201 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b509e7-1ca6-4587-a320-c9172e6c211a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:23.459634 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b509e7-1ca6-4587-a320-c9172e6c211a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:23.471999 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 332/683] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jun 15 19:25:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:23.477304 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b509e7-1ca6-4587-a320-c9172e6c211a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:23.499055 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851ba2d-6a63-4a5b-9fc0-4f7a23aea745 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:23.529079 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:23.536898 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462750c-ebc6-43db-9940-e1c2ecc1d1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:23.537558 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462750c-ebc6-43db-9940-e1c2ecc1d1de None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:23.537844 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462750c-ebc6-43db-9940-e1c2ecc1d1de None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:23.538142 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462750c-ebc6-43db-9940-e1c2ecc1d1de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:23.538980 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c462750c-ebc6-43db-9940-e1c2ecc1d1de None admin] RBAC: Adding query filter params (role.id=a8df9714563b435e9fce55a04e74ae5e, scope.project.id=7fb8cc0eb560465e972ffd45867ee73b, user.id=ec5882def40a4e0d8a792eecc12440a5) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:23.539292 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c462750c-ebc6-43db-9940-e1c2ecc1d1de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:23.540733 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c462750c-ebc6-43db-9940-e1c2ecc1d1de None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:23.562313 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 352/684] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:25:23 2022] GET /identity/v3/role_assignments?user.id=ec5882def40a4e0d8a792eecc12440a5&scope.project.id=7fb8cc0eb560465e972ffd45867ee73b&role.id=a8df9714563b435e9fce55a04e74ae5e => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:23.564256 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a06c5af-bdff-4d69-b95c-309f79b5aadf None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:23.564823 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a06c5af-bdff-4d69-b95c-309f79b5aadf None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:23.565417 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a06c5af-bdff-4d69-b95c-309f79b5aadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:23.588973 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-2a06c5af-bdff-4d69-b95c-309f79b5aadf None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:23.590508 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-2a06c5af-bdff-4d69-b95c-309f79b5aadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:23.613314 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b509e7-1ca6-4587-a320-c9172e6c211a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:23.668153 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce2078-eecf-40b3-909a-115bf97c601b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:23.669301 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce2078-eecf-40b3-909a-115bf97c601b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:23.669788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce2078-eecf-40b3-909a-115bf97c601b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:23.670316 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce2078-eecf-40b3-909a-115bf97c601b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:23.675373 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 353/685] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:25:23.675583 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a06c5af-bdff-4d69-b95c-309f79b5aadf None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:23.750636 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:23.759590 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ce2078-eecf-40b3-909a-115bf97c601b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:23.761101 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ce2078-eecf-40b3-909a-115bf97c601b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:23.812634 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:23.831778 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a06c5af-bdff-4d69-b95c-309f79b5aadf None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:23.869997 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5703c1-3025-44af-8619-50e8aa384836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:23.870691 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5703c1-3025-44af-8619-50e8aa384836 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:23.870969 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5703c1-3025-44af-8619-50e8aa384836 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:23.871240 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5703c1-3025-44af-8619-50e8aa384836 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:23.872071 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5703c1-3025-44af-8619-50e8aa384836 None admin] RBAC: Adding query filter params (role.id=a8df9714563b435e9fce55a04e74ae5e, scope.project.id=7fb8cc0eb560465e972ffd45867ee73b, user.id=5c63e0b6ceed4f7cb2e6f9269eb85795) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:23.872381 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5703c1-3025-44af-8619-50e8aa384836 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:23.873803 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5703c1-3025-44af-8619-50e8aa384836 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:23.880505 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 354/686] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:25:23 2022] GET /identity/v3/role_assignments?user.id=5c63e0b6ceed4f7cb2e6f9269eb85795&scope.project.id=7fb8cc0eb560465e972ffd45867ee73b&role.id=a8df9714563b435e9fce55a04e74ae5e => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:23.886115 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5703c1-3025-44af-8619-50e8aa384836 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:23.906225 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 333/687] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:25:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 19:25:23.910855 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce2078-eecf-40b3-909a-115bf97c601b None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:24.093574 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5703c1-3025-44af-8619-50e8aa384836 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:24.097886 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce2078-eecf-40b3-909a-115bf97c601b None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:24.110488 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:24.144183 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d42b8c-a7eb-407d-90cc-d9ea2a7f3a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:24.144982 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d42b8c-a7eb-407d-90cc-d9ea2a7f3a28 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:24.145260 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d42b8c-a7eb-407d-90cc-d9ea2a7f3a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:24.145532 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d42b8c-a7eb-407d-90cc-d9ea2a7f3a28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:24.149910 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f49353-9e29-45de-9de2-da92063e453c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:24.151110 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f49353-9e29-45de-9de2-da92063e453c None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:24.151618 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f49353-9e29-45de-9de2-da92063e453c None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:24.152730 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f49353-9e29-45de-9de2-da92063e453c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:24.157039 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d42b8c-a7eb-407d-90cc-d9ea2a7f3a28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:24.158532 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d42b8c-a7eb-407d-90cc-d9ea2a7f3a28 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:24.160345 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:24.168011 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f49353-9e29-45de-9de2-da92063e453c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:25:24.176573 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-04f49353-9e29-45de-9de2-da92063e453c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:25:24.176573 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:24.176573 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:24.176573 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:24.176573 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:24.176573 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:24.176573 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:24.176573 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:24.176573 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:24.176573 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:24.176573 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:24.176573 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:24.176573 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:25:24.176573 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:25:24.176573 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:25:24.176573 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:25:24.177491 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:25:24.177491 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:25:24.177491 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:25:24.177491 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:25:24.177491 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:24.177491 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:24.177491 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:24.177491 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:24.177491 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:24.177491 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:24.177491 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:24.177491 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:24.177491 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:24.177491 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:24.177491 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:24.177491 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:24.178364 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:24.178364 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:24.178364 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:24.178364 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:24.178364 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:25:24.178364 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:25:24.178364 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:25:24.178364 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:25:24.178364 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:25:24.178364 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:25:24.178364 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:25:24.178364 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:24.181340 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 355/688] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jun 15 19:25:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:24.186635 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-34d42b8c-a7eb-407d-90cc-d9ea2a7f3a28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:24.186635 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:24.186635 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:24.186635 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:24.186635 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:24.186635 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:24.186635 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:24.186635 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:24.186635 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:24.186635 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:24.186635 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:24.186635 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:24.186635 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:24.186635 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:24.186635 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:24.187548 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:24.187548 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:24.187548 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:24.187548 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:24.187548 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:24.187548 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:24.187548 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:24.187548 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:24.187548 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:24.187548 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:24.187548 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:24.187548 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:24.187548 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:24.187548 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:24.187548 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:24.187548 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:24.187548 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:24.188800 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:24.188800 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:24.188800 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:24.188800 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:24.188800 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 334/689] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:25:24.191145 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f49353-9e29-45de-9de2-da92063e453c None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:24.193443 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d42b8c-a7eb-407d-90cc-d9ea2a7f3a28 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:24.312412 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d42b8c-a7eb-407d-90cc-d9ea2a7f3a28 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:24.324449 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f49353-9e29-45de-9de2-da92063e453c None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:24.366062 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074e3f08-ac5b-40a2-8df8-b6ef02e6743d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:24.366697 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e3f08-ac5b-40a2-8df8-b6ef02e6743d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:24.367031 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e3f08-ac5b-40a2-8df8-b6ef02e6743d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:24.367314 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e3f08-ac5b-40a2-8df8-b6ef02e6743d None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:24.368046 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074e3f08-ac5b-40a2-8df8-b6ef02e6743d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:24.368352 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074e3f08-ac5b-40a2-8df8-b6ef02e6743d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:24.368828 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39279e6a-59d2-4e17-aa7c-2ddd38cd6ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:24.369789 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074e3f08-ac5b-40a2-8df8-b6ef02e6743d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:24.370350 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39279e6a-59d2-4e17-aa7c-2ddd38cd6ebf None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:24.370842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39279e6a-59d2-4e17-aa7c-2ddd38cd6ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:24.371413 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39279e6a-59d2-4e17-aa7c-2ddd38cd6ebf None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:24.372470 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39279e6a-59d2-4e17-aa7c-2ddd38cd6ebf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:24.372971 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39279e6a-59d2-4e17-aa7c-2ddd38cd6ebf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:24.374575 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39279e6a-59d2-4e17-aa7c-2ddd38cd6ebf None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:24.383851 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 335/690] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:24.384283 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 356/691] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:25:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:24.390368 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074e3f08-ac5b-40a2-8df8-b6ef02e6743d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:24.390744 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39279e6a-59d2-4e17-aa7c-2ddd38cd6ebf None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:24.442528 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:24.461235 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:24.512651 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074e3f08-ac5b-40a2-8df8-b6ef02e6743d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:24.549506 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e477ba30-51f0-416c-b396-386374b1178d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:24.550421 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477ba30-51f0-416c-b396-386374b1178d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:24.550757 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477ba30-51f0-416c-b396-386374b1178d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:24.551042 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477ba30-51f0-416c-b396-386374b1178d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:24.559544 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e477ba30-51f0-416c-b396-386374b1178d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:24.561477 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e477ba30-51f0-416c-b396-386374b1178d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:24.577617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-e477ba30-51f0-416c-b396-386374b1178d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:24.577617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:24.577617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:24.577617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:24.577617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:24.577617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:24.577617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:24.577617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:24.577617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:24.577617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:24.577617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:24.577617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:24.577617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 19:25:24.577617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 19:25:24.577617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 19:25:24.578570 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 19:25:24.578570 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:24.578570 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:24.578570 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:24.578570 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:24.578570 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:24.578570 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:24.578570 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:24.578570 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:24.578570 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:24.578570 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:24.578570 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:24.578570 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:24.578570 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:24.578570 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:24.578570 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:24.578570 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:24.579517 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 19:25:24.579517 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 19:25:24.579517 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:25:24.579517 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:25:24.579517 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:25:24.579517 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:25:24.579517 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:24.579517 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:24.583924 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 336/692] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:25:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:25:24.585312 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e477ba30-51f0-416c-b396-386374b1178d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:24.592018 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39279e6a-59d2-4e17-aa7c-2ddd38cd6ebf None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:24.682911 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1420ff44-a302-4293-a3e2-1a13b273c7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:24.683631 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420ff44-a302-4293-a3e2-1a13b273c7c4 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:24.683915 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420ff44-a302-4293-a3e2-1a13b273c7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:24.684189 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420ff44-a302-4293-a3e2-1a13b273c7c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:24.700734 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e477ba30-51f0-416c-b396-386374b1178d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:24.707185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1420ff44-a302-4293-a3e2-1a13b273c7c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:24.708649 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1420ff44-a302-4293-a3e2-1a13b273c7c4 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:24.717214 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:24.738695 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-1420ff44-a302-4293-a3e2-1a13b273c7c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:24.738695 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:24.738695 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:24.738695 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:24.738695 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:24.738695 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:24.738695 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:24.738695 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:24.738695 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:24.738695 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:24.738695 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:24.738695 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:24.738695 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:24.738695 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:24.738695 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:24.739648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:24.739648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:24.739648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:24.739648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:24.739648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:24.739648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:24.739648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:24.739648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:24.739648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:24.739648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:24.739648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:24.739648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:24.739648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:24.739648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:24.739648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:24.739648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:24.739648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:24.740578 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:24.740578 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:24.740578 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:24.740578 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:24.747031 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 357/693] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:25:24.752169 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1420ff44-a302-4293-a3e2-1a13b273c7c4 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:24.752366 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c3dc3-7987-4104-b869-1a7b8fd0388e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:24.753046 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c3dc3-7987-4104-b869-1a7b8fd0388e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:24.753333 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c3dc3-7987-4104-b869-1a7b8fd0388e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:24.753610 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c3dc3-7987-4104-b869-1a7b8fd0388e None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:24.754461 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028c3dc3-7987-4104-b869-1a7b8fd0388e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:24.754980 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028c3dc3-7987-4104-b869-1a7b8fd0388e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:24.756425 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028c3dc3-7987-4104-b869-1a7b8fd0388e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:24.776249 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 337/694] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jun 15 19:25:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:25:24.777786 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c3dc3-7987-4104-b869-1a7b8fd0388e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:24.925648 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c3dc3-7987-4104-b869-1a7b8fd0388e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:24.960917 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:24.968972 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec21219f-0c11-4066-b288-bb4feb4ec759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:24.969642 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec21219f-0c11-4066-b288-bb4feb4ec759 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:24.969926 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec21219f-0c11-4066-b288-bb4feb4ec759 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:24.970303 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec21219f-0c11-4066-b288-bb4feb4ec759 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:24.971104 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec21219f-0c11-4066-b288-bb4feb4ec759 None admin] RBAC: Adding query filter params (role.id=a8df9714563b435e9fce55a04e74ae5e, scope.project.id=7fb8cc0eb560465e972ffd45867ee73b, user.id=176d5775a70f4524a9b7fe1ce9d426ed) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:24.971415 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec21219f-0c11-4066-b288-bb4feb4ec759 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:24.972851 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec21219f-0c11-4066-b288-bb4feb4ec759 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:24.984323 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 338/695] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:25:24 2022] GET /identity/v3/role_assignments?user.id=176d5775a70f4524a9b7fe1ce9d426ed&scope.project.id=7fb8cc0eb560465e972ffd45867ee73b&role.id=a8df9714563b435e9fce55a04e74ae5e => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:24.995738 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1420ff44-a302-4293-a3e2-1a13b273c7c4 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:25.041063 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecc0cff-3953-407f-a8a2-40784bdb1b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:25.042189 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc0cff-3953-407f-a8a2-40784bdb1b11 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:25.042674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc0cff-3953-407f-a8a2-40784bdb1b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:25.043146 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc0cff-3953-407f-a8a2-40784bdb1b11 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:25.044091 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecc0cff-3953-407f-a8a2-40784bdb1b11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:25.044604 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecc0cff-3953-407f-a8a2-40784bdb1b11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:25.046270 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecc0cff-3953-407f-a8a2-40784bdb1b11 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:25.073165 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 358/696] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:25:25.078986 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec21219f-0c11-4066-b288-bb4feb4ec759 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:25.135652 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:25.176536 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3f3e6-4824-4d63-9019-5eaa7cf8c26b None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:25.177176 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3f3e6-4824-4d63-9019-5eaa7cf8c26b None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:25.177686 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3f3e6-4824-4d63-9019-5eaa7cf8c26b None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:25.179395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 359/697] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 19:25:25.184720 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e90989-fb8b-4538-8349-7e1f94357669 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:25.185010 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e90989-fb8b-4538-8349-7e1f94357669 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:25.185287 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e90989-fb8b-4538-8349-7e1f94357669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:25.209712 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-66e90989-fb8b-4538-8349-7e1f94357669 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:25.210979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-66e90989-fb8b-4538-8349-7e1f94357669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:25.213734 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec21219f-0c11-4066-b288-bb4feb4ec759 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:25.252120 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f845d0ab-1513-4d43-8416-e29e75270fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:25.253359 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845d0ab-1513-4d43-8416-e29e75270fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:25.253975 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845d0ab-1513-4d43-8416-e29e75270fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:25.254487 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845d0ab-1513-4d43-8416-e29e75270fd7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:25.266915 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f845d0ab-1513-4d43-8416-e29e75270fd7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:25.268733 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f845d0ab-1513-4d43-8416-e29e75270fd7 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:25.284155 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-f845d0ab-1513-4d43-8416-e29e75270fd7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 19:25:25.284155 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:25.284155 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:25.284155 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:25.284155 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:25.284155 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:25.284155 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:25.284155 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:25.284155 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:25.284155 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:25.284155 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:25.284155 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:25.284155 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 19:25:25.284155 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 19:25:25.284155 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 19:25:25.284155 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 19:25:25.285130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:25.285130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:25.285130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:25:25.285130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:25.285130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:25:25.285130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:25.285130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:25.285130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:25.285130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:25.285130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:25.285130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:25.285130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:25.285130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:25.285130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:25.285130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:25.285130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:25.285130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:25.285130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:25.286109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:25.286109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:25.286109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 19:25:25.286109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 19:25:25.286109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 19:25:25.286109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 19:25:25.286109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 19:25:25.286109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 19:25:25.286109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 19:25:25.286109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:25.288688 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 339/698] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Jun 15 19:25:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:25.292244 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 360/699] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:25:25.294845 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f845d0ab-1513-4d43-8416-e29e75270fd7 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:25.321063 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e90989-fb8b-4538-8349-7e1f94357669 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:25.410437 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:25.451026 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e90989-fb8b-4538-8349-7e1f94357669 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:25.453989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:25.459363 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f845d0ab-1513-4d43-8416-e29e75270fd7 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:25.497604 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7f8c4-b15b-4e47-a896-0a3836121e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:25.498843 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7f8c4-b15b-4e47-a896-0a3836121e8d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:25.499277 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7f8c4-b15b-4e47-a896-0a3836121e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:25.499834 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7f8c4-b15b-4e47-a896-0a3836121e8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:25.505462 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8a169-d209-4c7b-9d57-8b058ec51f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:25.506266 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8a169-d209-4c7b-9d57-8b058ec51f84 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:25.506621 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8a169-d209-4c7b-9d57-8b058ec51f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:25.506974 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8a169-d209-4c7b-9d57-8b058ec51f84 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:25.507897 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8a169-d209-4c7b-9d57-8b058ec51f84 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:25.508556 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8a169-d209-4c7b-9d57-8b058ec51f84 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:25.510691 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8a169-d209-4c7b-9d57-8b058ec51f84 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:25.514801 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad7f8c4-b15b-4e47-a896-0a3836121e8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:25.516524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad7f8c4-b15b-4e47-a896-0a3836121e8d None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:25.532181 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 340/700] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Jun 15 19:25:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:25.537226 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8a169-d209-4c7b-9d57-8b058ec51f84 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:25.538388 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-8ad7f8c4-b15b-4e47-a896-0a3836121e8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:25.538388 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:25.538388 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:25.538388 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:25.538388 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:25.538388 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:25.538388 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:25.538388 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:25.538388 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:25.538388 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:25.538388 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:25.538388 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:25.538388 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:25.538388 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:25.538388 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:25.539274 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:25.539274 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:25.539274 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:25.539274 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:25.539274 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:25.539274 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:25.539274 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:25.539274 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:25.539274 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:25.539274 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:25.539274 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:25.539274 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:25.539274 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:25.539274 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:25.539274 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:25.539274 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:25.539274 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:25.540170 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:25.540170 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:25.540170 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:25.540170 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:25.540170 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 361/701] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:25:25.547782 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7f8c4-b15b-4e47-a896-0a3836121e8d None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:25.660914 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7f8c4-b15b-4e47-a896-0a3836121e8d None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:25.674682 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8a169-d209-4c7b-9d57-8b058ec51f84 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:25.700152 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:25.720205 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06745c4-a9af-4038-a5eb-672286e18bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:25.720872 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06745c4-a9af-4038-a5eb-672286e18bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:25.721150 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06745c4-a9af-4038-a5eb-672286e18bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:25.721420 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06745c4-a9af-4038-a5eb-672286e18bc3 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:25.723711 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06745c4-a9af-4038-a5eb-672286e18bc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:25.724026 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06745c4-a9af-4038-a5eb-672286e18bc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:25.725403 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06745c4-a9af-4038-a5eb-672286e18bc3 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:25.740620 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c27a5-d545-4455-86ab-90877507ee16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:25.741017 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 362/702] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:25:25.741929 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c27a5-d545-4455-86ab-90877507ee16 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:25.750187 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c27a5-d545-4455-86ab-90877507ee16 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:25.750313 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06745c4-a9af-4038-a5eb-672286e18bc3 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:25.750393 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c27a5-d545-4455-86ab-90877507ee16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:25.799755 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:25.879346 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320c27a5-d545-4455-86ab-90877507ee16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:25.880825 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320c27a5-d545-4455-86ab-90877507ee16 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:25.942355 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06745c4-a9af-4038-a5eb-672286e18bc3 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:26.011580 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d741c5f-98fc-438a-b4c7-776e69e03289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:26.012243 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d741c5f-98fc-438a-b4c7-776e69e03289 None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:26.012647 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d741c5f-98fc-438a-b4c7-776e69e03289 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:26.012923 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d741c5f-98fc-438a-b4c7-776e69e03289 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:26.013629 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d741c5f-98fc-438a-b4c7-776e69e03289 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:26.015179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d741c5f-98fc-438a-b4c7-776e69e03289 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:26.040447 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 363/703] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Jun 15 19:25:25 2022] POST /identity/v3/users => generated 254 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:25:26.041932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd6d05-a687-47c3-a2be-8192ef90f7ec None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:26.046624 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd6d05-a687-47c3-a2be-8192ef90f7ec None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:26.046904 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd6d05-a687-47c3-a2be-8192ef90f7ec None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:26.047374 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 341/704] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:25:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 19:25:26.048644 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 364/705] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:26.048771 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c27a5-d545-4455-86ab-90877507ee16 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:26.063012 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217ce45-df9e-4845-8cc5-3186416fb893 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:26.063322 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217ce45-df9e-4845-8cc5-3186416fb893 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:26.063578 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217ce45-df9e-4845-8cc5-3186416fb893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:26.101625 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-c217ce45-df9e-4845-8cc5-3186416fb893 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:26.103000 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-c217ce45-df9e-4845-8cc5-3186416fb893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:26.118480 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:26.170695 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c27a5-d545-4455-86ab-90877507ee16 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:26.208138 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:26.209895 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d284e0-b1e0-47b0-80b4-44ca7a1af735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:26.210921 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d284e0-b1e0-47b0-80b4-44ca7a1af735 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:26.211204 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d284e0-b1e0-47b0-80b4-44ca7a1af735 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:26.211479 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d284e0-b1e0-47b0-80b4-44ca7a1af735 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:26.224658 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d284e0-b1e0-47b0-80b4-44ca7a1af735 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:26.226373 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d284e0-b1e0-47b0-80b4-44ca7a1af735 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:26.247442 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-b8d284e0-b1e0-47b0-80b4-44ca7a1af735 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:26.247442 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:26.247442 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:26.247442 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:26.247442 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:26.247442 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:26.247442 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:26.247442 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:26.247442 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:26.247442 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:26.247442 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:26.247442 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:26.247442 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:26.247442 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:26.247442 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:26.248400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:26.248400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:26.248400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:26.248400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:26.248400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:26.248400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:26.248400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:26.248400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:26.248400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:26.248400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:26.248400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:26.248400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:26.248400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:26.248400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:26.248400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:26.248400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:26.248400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:26.249325 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:26.249325 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:26.249325 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:26.249325 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:26.249325 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 342/706] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:25:26.255152 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d284e0-b1e0-47b0-80b4-44ca7a1af735 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:26.287920 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 365/707] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 19:25:26.310635 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c217ce45-df9e-4845-8cc5-3186416fb893 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:26.367793 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d284e0-b1e0-47b0-80b4-44ca7a1af735 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:26.377592 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:26.405256 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238e44e-ac87-4e62-ad79-053fc1a32b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:26.405923 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238e44e-ac87-4e62-ad79-053fc1a32b1a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:26.406235 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238e44e-ac87-4e62-ad79-053fc1a32b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:26.406507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238e44e-ac87-4e62-ad79-053fc1a32b1a None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:26.407228 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238e44e-ac87-4e62-ad79-053fc1a32b1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:26.407531 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238e44e-ac87-4e62-ad79-053fc1a32b1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:26.408900 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238e44e-ac87-4e62-ad79-053fc1a32b1a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:26.420040 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c217ce45-df9e-4845-8cc5-3186416fb893 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:26.420602 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 343/708] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:26.428480 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238e44e-ac87-4e62-ad79-053fc1a32b1a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:26.475244 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7a44ec-b63a-439b-9694-725866602dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:26.476069 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a44ec-b63a-439b-9694-725866602dad None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:26.476353 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a44ec-b63a-439b-9694-725866602dad None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:26.476630 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a44ec-b63a-439b-9694-725866602dad None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:26.477295 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7a44ec-b63a-439b-9694-725866602dad None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:26.478861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7a44ec-b63a-439b-9694-725866602dad None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:26.497476 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-4a7a44ec-b63a-439b-9694-725866602dad None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 19:25:26.497476 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:26.497476 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 19:25:26.497476 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 19:25:26.497476 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 19:25:26.497476 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 19:25:26.497476 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 19:25:26.497476 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:25:26.497476 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 19:25:26.497476 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:26.497476 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 19:25:26.497476 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:26.497476 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:26.497476 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:26.497476 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:26.497476 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:26.498521 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:26.498521 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:26.498521 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:26.498521 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:26.498521 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:26.498521 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:26.498521 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:26.498521 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 19:25:26.498521 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 19:25:26.498521 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 19:25:26.498521 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 19:25:26.498521 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:26.498521 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:26.498521 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:26.498521 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:26.498521 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:26.499375 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:26.499375 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:26.499375 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:26.499375 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:26.499375 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:26.499375 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:26.499375 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:26.499375 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:26.499375 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:26.499375 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:26.499375 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:26.499375 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 19:25:26.499375 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:25:26.499375 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 19:25:26.499375 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:26.502433 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 366/709] 10.222.0.15 () {56 vars in 1129 bytes} [Wed Jun 15 19:25:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:25:26.504248 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7a44ec-b63a-439b-9694-725866602dad None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:26.545501 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:26.561642 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238e44e-ac87-4e62-ad79-053fc1a32b1a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:26.602408 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d06369-c363-4700-9fcd-c801155c48a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:26.604087 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d06369-c363-4700-9fcd-c801155c48a4 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:26.604507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d06369-c363-4700-9fcd-c801155c48a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:26.604925 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d06369-c363-4700-9fcd-c801155c48a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:26.612259 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d06369-c363-4700-9fcd-c801155c48a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:26.613947 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d06369-c363-4700-9fcd-c801155c48a4 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:26.624579 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7a44ec-b63a-439b-9694-725866602dad None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:26.630798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-93d06369-c363-4700-9fcd-c801155c48a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:26.630798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:26.630798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:26.630798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:26.630798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:26.630798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:26.630798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:26.630798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:26.630798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:26.630798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:26.630798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:26.630798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:26.630798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 19:25:26.630798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 19:25:26.630798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 19:25:26.631866 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 19:25:26.631866 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:26.631866 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:26.631866 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:26.631866 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:26.631866 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:26.631866 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:26.631866 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:26.631866 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:26.631866 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:26.631866 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:26.631866 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:26.631866 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:26.631866 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:26.631866 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:26.631866 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:26.631866 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:26.632949 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 19:25:26.632949 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 19:25:26.632949 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:25:26.632949 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:25:26.632949 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:25:26.632949 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:25:26.632949 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:26.632949 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:26.635382 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 344/710] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:25:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:25:26.641151 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d06369-c363-4700-9fcd-c801155c48a4 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:26.663358 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f88277-9f43-4575-8066-54e203ad523b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:26.664094 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f88277-9f43-4575-8066-54e203ad523b None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:26.664711 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f88277-9f43-4575-8066-54e203ad523b None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:26.664711 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f88277-9f43-4575-8066-54e203ad523b None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:26.665496 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f88277-9f43-4575-8066-54e203ad523b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:26.665852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f88277-9f43-4575-8066-54e203ad523b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:26.667837 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f88277-9f43-4575-8066-54e203ad523b None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:26.672168 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:26.678241 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 367/711] 10.222.0.15 () {56 vars in 1124 bytes} [Wed Jun 15 19:25:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:25:26.679875 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f88277-9f43-4575-8066-54e203ad523b None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:26.750671 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d06369-c363-4700-9fcd-c801155c48a4 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:26.787259 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb2b473-652c-47f9-88b0-ad1388e207a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:26.788067 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2b473-652c-47f9-88b0-ad1388e207a4 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:26.788210 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2b473-652c-47f9-88b0-ad1388e207a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:26.788469 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2b473-652c-47f9-88b0-ad1388e207a4 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:26.789329 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb2b473-652c-47f9-88b0-ad1388e207a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:26.789591 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb2b473-652c-47f9-88b0-ad1388e207a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:26.791249 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb2b473-652c-47f9-88b0-ad1388e207a4 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:26.798688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f88277-9f43-4575-8066-54e203ad523b None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:26.802285 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:26.803802 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 345/712] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jun 15 19:25:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:26.810131 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb2b473-652c-47f9-88b0-ad1388e207a4 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:26.838987 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc60cc8b-8dfa-4200-87bb-19316bfb827b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:26.840176 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc60cc8b-8dfa-4200-87bb-19316bfb827b None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:26.840915 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc60cc8b-8dfa-4200-87bb-19316bfb827b None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:26.841370 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc60cc8b-8dfa-4200-87bb-19316bfb827b None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:26.842256 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc60cc8b-8dfa-4200-87bb-19316bfb827b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:26.842809 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc60cc8b-8dfa-4200-87bb-19316bfb827b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:26.844480 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc60cc8b-8dfa-4200-87bb-19316bfb827b None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:26.858263 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 368/713] 10.222.0.15 () {56 vars in 1124 bytes} [Wed Jun 15 19:25:26 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:26.872936 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166ccef-09fc-4719-8a2f-d05cb66a34be None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:26.873456 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166ccef-09fc-4719-8a2f-d05cb66a34be None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:26.873908 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166ccef-09fc-4719-8a2f-d05cb66a34be None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:26.876542 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 369/714] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 19:25:26.880804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7468f1a-50c8-4865-a86d-5a895dad7926 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:26.881331 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7468f1a-50c8-4865-a86d-5a895dad7926 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:26.881821 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7468f1a-50c8-4865-a86d-5a895dad7926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:26.915458 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-d7468f1a-50c8-4865-a86d-5a895dad7926 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:26.917533 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-d7468f1a-50c8-4865-a86d-5a895dad7926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:26.919405 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:26.933434 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb2b473-652c-47f9-88b0-ad1388e207a4 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:26.972529 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f24096-bf30-40ab-a720-f018c72017f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:26.973562 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f24096-bf30-40ab-a720-f018c72017f2 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:26.973562 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f24096-bf30-40ab-a720-f018c72017f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:26.973759 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f24096-bf30-40ab-a720-f018c72017f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:26.974645 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f24096-bf30-40ab-a720-f018c72017f2 None admin] RBAC: Adding query filter params (role.id=4f736313c26d4f8db48bbb2f346e50e4, scope.project.id=7fb8cc0eb560465e972ffd45867ee73b, user.id=3e74ac44e5814122a3226ac023f3688a) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:26.975037 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f24096-bf30-40ab-a720-f018c72017f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:26.976406 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f24096-bf30-40ab-a720-f018c72017f2 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:26.983688 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 346/715] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:25:26 2022] GET /identity/v3/role_assignments?user.id=3e74ac44e5814122a3226ac023f3688a&scope.project.id=7fb8cc0eb560465e972ffd45867ee73b&role.id=4f736313c26d4f8db48bbb2f346e50e4 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:27.052720 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 370/716] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 19:25:27.085634 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f24096-bf30-40ab-a720-f018c72017f2 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:27.208164 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f24096-bf30-40ab-a720-f018c72017f2 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:27.246166 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6415bdb7-c378-464a-a0cb-ac2e1ceef176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:27.247231 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415bdb7-c378-464a-a0cb-ac2e1ceef176 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:27.247685 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415bdb7-c378-464a-a0cb-ac2e1ceef176 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:27.248107 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415bdb7-c378-464a-a0cb-ac2e1ceef176 None admin] PATH_INFO: `/v3/services/network` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:27.248914 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6415bdb7-c378-464a-a0cb-ac2e1ceef176 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:27.250509 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6415bdb7-c378-464a-a0cb-ac2e1ceef176 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:27.265772 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-6415bdb7-c378-464a-a0cb-ac2e1ceef176 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 19:25:27.265772 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:27.265772 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 19:25:27.265772 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 19:25:27.265772 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 19:25:27.265772 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 19:25:27.265772 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 19:25:27.265772 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:25:27.265772 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 19:25:27.265772 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:27.265772 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 19:25:27.265772 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:27.265772 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:27.265772 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:27.265772 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:27.265772 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:27.265772 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:27.267023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:27.267023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:27.267023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:27.267023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:27.267023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:27.267023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:27.267023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 19:25:27.267023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 19:25:27.267023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 19:25:27.267023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 19:25:27.267023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:27.267023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:27.267023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:27.267023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:27.267023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:27.267023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:27.268057 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:27.268057 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:27.268057 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:27.268057 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:27.268057 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:27.268057 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:27.268057 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:27.268057 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:27.268057 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:27.268057 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:27.268057 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 19:25:27.268057 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:25:27.268057 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 19:25:27.268057 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:27.270725 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 347/717] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:25:27 2022] GET /identity/v3/services/network => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:27.277339 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7468f1a-50c8-4865-a86d-5a895dad7926 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:27.429589 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:27.438627 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7468f1a-50c8-4865-a86d-5a895dad7926 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:27.490947 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02725c02-91e0-40f1-aa36-8c527845e320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:27.491619 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02725c02-91e0-40f1-aa36-8c527845e320 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:27.491903 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02725c02-91e0-40f1-aa36-8c527845e320 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:27.492178 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02725c02-91e0-40f1-aa36-8c527845e320 None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:27.492891 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02725c02-91e0-40f1-aa36-8c527845e320 None admin] RBAC: Adding query filter params (name=network) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:27.493198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02725c02-91e0-40f1-aa36-8c527845e320 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:27.495158 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02725c02-91e0-40f1-aa36-8c527845e320 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:27.506007 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 371/718] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Jun 15 19:25:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:27.512276 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6415bdb7-c378-464a-a0cb-ac2e1ceef176 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:27.542312 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:27.645703 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6415bdb7-c378-464a-a0cb-ac2e1ceef176 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:27.693543 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff892438-9cad-4a19-a282-f0ac65c96b02 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:27.693981 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff892438-9cad-4a19-a282-f0ac65c96b02 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:27.694509 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff892438-9cad-4a19-a282-f0ac65c96b02 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:27.695781 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d07cd1-7611-4ab1-ba97-f176f9125471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:27.696327 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 372/719] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:27.696801 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d07cd1-7611-4ab1-ba97-f176f9125471 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:27.697779 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d07cd1-7611-4ab1-ba97-f176f9125471 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:27.698220 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d07cd1-7611-4ab1-ba97-f176f9125471 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:27.699086 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d07cd1-7611-4ab1-ba97-f176f9125471 None admin] RBAC: Adding query filter params (type=network) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:27.699528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d07cd1-7611-4ab1-ba97-f176f9125471 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:27.701070 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d07cd1-7611-4ab1-ba97-f176f9125471 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:27.702192 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c31817-cda1-468c-9046-487d63eb82fc None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:27.702485 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c31817-cda1-468c-9046-487d63eb82fc None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:27.702766 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c31817-cda1-468c-9046-487d63eb82fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:27.711158 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 348/720] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Jun 15 19:25:27 2022] GET /identity/v3/services?type=network => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:27.734674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-c5c31817-cda1-468c-9046-487d63eb82fc None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:27.736216 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-c5c31817-cda1-468c-9046-487d63eb82fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:27.844803 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 373/721] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:25:27.884224 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d07cd1-7611-4ab1-ba97-f176f9125471 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:27.993290 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:28.017553 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d07cd1-7611-4ab1-ba97-f176f9125471 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:28.053269 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec9506f-bb96-4b68-9455-88a038d6b7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:28.054451 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9506f-bb96-4b68-9455-88a038d6b7f1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:28.054933 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9506f-bb96-4b68-9455-88a038d6b7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:28.055402 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9506f-bb96-4b68-9455-88a038d6b7f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:28.066810 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec9506f-bb96-4b68-9455-88a038d6b7f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:25:28.074956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-eec9506f-bb96-4b68-9455-88a038d6b7f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:25:28.074956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:28.074956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:28.074956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:28.074956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:28.074956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:28.074956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:28.074956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:28.074956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:28.074956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:28.074956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:28.074956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:28.074956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:25:28.074956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:25:28.074956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:25:28.074956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:25:28.077084 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:25:28.077084 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:25:28.077084 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:25:28.077084 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:25:28.077084 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:28.077084 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:28.077084 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:28.077084 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:28.077084 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:28.077084 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:28.077084 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:28.077084 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:28.077084 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:28.077084 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:28.077084 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:28.077084 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:28.077921 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:28.077921 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:28.077921 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:28.077921 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:28.077921 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:25:28.077921 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:25:28.077921 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:25:28.077921 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:25:28.077921 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:25:28.077921 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:25:28.077921 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:25:28.077921 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:28.079610 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 349/722] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jun 15 19:25:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:28.085820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c31817-cda1-468c-9046-487d63eb82fc None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:28.225094 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:28.241632 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c31817-cda1-468c-9046-487d63eb82fc None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:28.287364 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea99b39-d7e6-47d2-903d-240921439840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:28.287963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea99b39-d7e6-47d2-903d-240921439840 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:28.288246 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea99b39-d7e6-47d2-903d-240921439840 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:28.288523 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea99b39-d7e6-47d2-903d-240921439840 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:28.289275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea99b39-d7e6-47d2-903d-240921439840 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:28.289582 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea99b39-d7e6-47d2-903d-240921439840 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:28.291566 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea99b39-d7e6-47d2-903d-240921439840 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:28.307199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 374/723] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:25:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:28.312427 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec9506f-bb96-4b68-9455-88a038d6b7f1 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:28.433429 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec9506f-bb96-4b68-9455-88a038d6b7f1 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:28.470015 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f2046-56d9-4d36-adc7-3c54a3034271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:28.470842 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f2046-56d9-4d36-adc7-3c54a3034271 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:28.471122 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f2046-56d9-4d36-adc7-3c54a3034271 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:28.471401 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f2046-56d9-4d36-adc7-3c54a3034271 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:28.481003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:28.483945 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36f2046-56d9-4d36-adc7-3c54a3034271 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:28.485373 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36f2046-56d9-4d36-adc7-3c54a3034271 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:28.503596 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-e36f2046-56d9-4d36-adc7-3c54a3034271 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:28.503596 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:28.503596 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:28.503596 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:28.503596 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:28.503596 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:28.503596 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:28.503596 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:28.503596 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:28.503596 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:28.503596 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:28.503596 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:28.503596 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:28.503596 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:28.503596 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:28.504530 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:28.504530 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:28.504530 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:28.504530 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:28.504530 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:28.504530 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:28.504530 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:28.504530 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:28.504530 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:28.504530 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:28.504530 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:28.504530 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:28.504530 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:28.504530 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:28.504530 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:28.504530 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:28.504530 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:28.506143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:28.506143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:28.506143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:28.506143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:28.506143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 350/724] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:28.510940 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea99b39-d7e6-47d2-903d-240921439840 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:28.674060 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea99b39-d7e6-47d2-903d-240921439840 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:28.725375 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3652f22-8eb2-47de-83a4-2a0b928411af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:28.726045 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3652f22-8eb2-47de-83a4-2a0b928411af None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:28.726355 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3652f22-8eb2-47de-83a4-2a0b928411af None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:28.726630 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3652f22-8eb2-47de-83a4-2a0b928411af None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:28.727350 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3652f22-8eb2-47de-83a4-2a0b928411af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:28.727655 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3652f22-8eb2-47de-83a4-2a0b928411af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:28.729028 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3652f22-8eb2-47de-83a4-2a0b928411af None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:28.742241 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 375/725] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:28.751512 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f2046-56d9-4d36-adc7-3c54a3034271 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:28.912649 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f2046-56d9-4d36-adc7-3c54a3034271 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:28.974985 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc49aa5-e7dd-455d-bda2-ee5c9d27e572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:28.975705 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc49aa5-e7dd-455d-bda2-ee5c9d27e572 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:28.975986 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc49aa5-e7dd-455d-bda2-ee5c9d27e572 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:28.976263 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc49aa5-e7dd-455d-bda2-ee5c9d27e572 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:28.991176 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc49aa5-e7dd-455d-bda2-ee5c9d27e572 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:28.992621 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc49aa5-e7dd-455d-bda2-ee5c9d27e572 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:29.011181 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-bcc49aa5-e7dd-455d-bda2-ee5c9d27e572 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 19:25:29.011181 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:29.011181 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:29.011181 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:29.011181 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:29.011181 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:29.011181 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:29.011181 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:29.011181 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:29.011181 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:29.011181 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:29.011181 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:29.011181 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 19:25:29.011181 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 19:25:29.011181 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 19:25:29.011181 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 19:25:29.013595 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:29.013595 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:29.013595 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:25:29.013595 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:29.013595 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:25:29.013595 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:29.013595 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:29.013595 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:29.013595 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:29.013595 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:29.013595 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:29.013595 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:29.013595 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:29.013595 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:29.013595 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:29.013595 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:29.013595 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:29.013595 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:29.015908 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:29.015908 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:29.015908 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 19:25:29.015908 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 19:25:29.015908 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 19:25:29.015908 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 19:25:29.015908 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 19:25:29.015908 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 19:25:29.015908 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 19:25:29.015908 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:29.015908 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 351/726] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jun 15 19:25:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:29.020727 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3652f22-8eb2-47de-83a4-2a0b928411af None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:29.042007 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:29.197169 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3652f22-8eb2-47de-83a4-2a0b928411af None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:29.219930 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ac88c-3c11-4de5-9921-340eb287929c None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:29.220245 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ac88c-3c11-4de5-9921-340eb287929c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:29.220489 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ac88c-3c11-4de5-9921-340eb287929c None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:29.222213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 352/727] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:29.227105 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bdf5e4-eb9d-403c-92d7-4049ed251ffc None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:29.227331 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bdf5e4-eb9d-403c-92d7-4049ed251ffc None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:29.227607 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bdf5e4-eb9d-403c-92d7-4049ed251ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:29.243338 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2fbba-36f5-4092-9429-e0b913638fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:29.244971 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2fbba-36f5-4092-9429-e0b913638fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:29.245477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2fbba-36f5-4092-9429-e0b913638fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:29.245949 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2fbba-36f5-4092-9429-e0b913638fe0 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:29.247185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c2fbba-36f5-4092-9429-e0b913638fe0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:29.247721 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c2fbba-36f5-4092-9429-e0b913638fe0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:29.249353 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c2fbba-36f5-4092-9429-e0b913638fe0 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:29.269088 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 376/728] 10.222.0.15 () {56 vars in 1135 bytes} [Wed Jun 15 19:25:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:29.271282 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-49bdf5e4-eb9d-403c-92d7-4049ed251ffc None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:29.272781 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-49bdf5e4-eb9d-403c-92d7-4049ed251ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:29.276720 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2fbba-36f5-4092-9429-e0b913638fe0 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:29.358490 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:29.371384 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:29.399143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 353/729] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:25:29.425123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2fbba-36f5-4092-9429-e0b913638fe0 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:29.440848 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bdf5e4-eb9d-403c-92d7-4049ed251ffc None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:29.486694 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f1a4a-45ab-44af-a50a-d8edf0020e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:29.487593 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f1a4a-45ab-44af-a50a-d8edf0020e58 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:29.488018 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f1a4a-45ab-44af-a50a-d8edf0020e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:29.489010 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f1a4a-45ab-44af-a50a-d8edf0020e58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:29.612014 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f1a4a-45ab-44af-a50a-d8edf0020e58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:29.614177 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f1a4a-45ab-44af-a50a-d8edf0020e58 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:29.636653 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bdf5e4-eb9d-403c-92d7-4049ed251ffc None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:29.673614 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a36b9-535a-455e-9558-5b232c2e8e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:29.674438 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a36b9-535a-455e-9558-5b232c2e8e2a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:29.674720 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a36b9-535a-455e-9558-5b232c2e8e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:29.674996 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a36b9-535a-455e-9558-5b232c2e8e2a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:29.686118 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102a36b9-535a-455e-9558-5b232c2e8e2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:25:29.689081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:29.693890 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-102a36b9-535a-455e-9558-5b232c2e8e2a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:25:29.693890 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:29.693890 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:29.693890 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:29.693890 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:29.693890 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:29.693890 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:29.693890 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:29.693890 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:29.693890 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:29.693890 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:29.693890 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:29.693890 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:25:29.693890 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:25:29.693890 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:25:29.693890 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:25:29.695033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:25:29.695033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:25:29.695033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:25:29.695033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:25:29.695033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:29.695033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:29.695033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:29.695033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:29.695033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:29.695033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:29.695033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:29.695033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:29.695033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:29.695033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:29.695033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:29.695033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:29.696143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:29.696143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:29.696143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:29.696143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:29.696143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:25:29.696143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:25:29.696143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:25:29.696143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:25:29.696143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:25:29.696143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:25:29.696143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:25:29.696143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:29.698164 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 354/730] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jun 15 19:25:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:25:29.698494 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36581a5-270f-4e1d-bab8-1cf083147ae0 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:29.698785 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36581a5-270f-4e1d-bab8-1cf083147ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:29.699065 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36581a5-270f-4e1d-bab8-1cf083147ae0 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:29.700376 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 355/731] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:29.701465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e9670-acd8-452a-9a3b-a73684488ec3 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:29.701719 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e9670-acd8-452a-9a3b-a73684488ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:29.702026 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e9670-acd8-452a-9a3b-a73684488ec3 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:29.703399 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 356/732] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:29.707510 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e9670-acd8-452a-9a3b-a73684488ec3 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:29.739422 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 377/733] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:25:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 19:25:29.740995 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66c329-af02-4017-9ade-0976588cee7c None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:29.741286 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66c329-af02-4017-9ade-0976588cee7c None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:29.741563 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66c329-af02-4017-9ade-0976588cee7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:29.757789 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-6f66c329-af02-4017-9ade-0976588cee7c None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:29.758991 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-6f66c329-af02-4017-9ade-0976588cee7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:29.763324 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:29.826467 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e9670-acd8-452a-9a3b-a73684488ec3 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:29.830409 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 378/734] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:25:29.831735 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276247aa-6116-4562-ab57-ca7c92b9c766 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:29.832026 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276247aa-6116-4562-ab57-ca7c92b9c766 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:29.832301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276247aa-6116-4562-ab57-ca7c92b9c766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:29.848782 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-276247aa-6116-4562-ab57-ca7c92b9c766 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:29.849955 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-276247aa-6116-4562-ab57-ca7c92b9c766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:29.873052 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383499bd-c12f-4569-937d-bd3b2667e9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:29.873724 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383499bd-c12f-4569-937d-bd3b2667e9ee None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:29.874039 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383499bd-c12f-4569-937d-bd3b2667e9ee None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:29.874308 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383499bd-c12f-4569-937d-bd3b2667e9ee None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:29.875063 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383499bd-c12f-4569-937d-bd3b2667e9ee None admin] RBAC: Adding query filter params (name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:29.875370 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383499bd-c12f-4569-937d-bd3b2667e9ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:29.876777 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383499bd-c12f-4569-937d-bd3b2667e9ee None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:29.889510 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 357/735] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jun 15 19:25:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:29.889510 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383499bd-c12f-4569-937d-bd3b2667e9ee None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:29.926602 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 379/736] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:25:29.927731 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276247aa-6116-4562-ab57-ca7c92b9c766 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:29.960206 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:29.998462 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383499bd-c12f-4569-937d-bd3b2667e9ee None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:30.001692 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:30.034053 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f2532e-b48e-4ac9-b23f-644cc71c12f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:30.034865 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f2532e-b48e-4ac9-b23f-644cc71c12f5 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:30.035150 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f2532e-b48e-4ac9-b23f-644cc71c12f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:30.035426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f2532e-b48e-4ac9-b23f-644cc71c12f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:30.039692 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276247aa-6116-4562-ab57-ca7c92b9c766 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:30.045745 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f2532e-b48e-4ac9-b23f-644cc71c12f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:30.047226 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f2532e-b48e-4ac9-b23f-644cc71c12f5 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:30.064750 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-f3f2532e-b48e-4ac9-b23f-644cc71c12f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:30.064750 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:30.064750 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:30.064750 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:30.064750 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:30.064750 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:30.064750 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:30.064750 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:30.064750 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:30.064750 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:30.064750 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:30.064750 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:30.064750 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:30.064750 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:30.064750 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:30.065661 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:30.065661 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:30.065661 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:30.065661 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:30.065661 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:30.065661 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:30.065661 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:30.065661 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:30.065661 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:30.065661 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:30.065661 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:30.065661 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:30.065661 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:30.065661 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:30.065661 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:30.065661 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:30.065661 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:30.066680 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:30.066680 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:30.066680 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:30.066680 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:30.066680 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 358/737] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:30.066958 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f2532e-b48e-4ac9-b23f-644cc71c12f5 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:30.082301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c31a9bd-5ac3-4719-8c23-2609f64afbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:30.082981 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31a9bd-5ac3-4719-8c23-2609f64afbb1 None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:30.083267 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31a9bd-5ac3-4719-8c23-2609f64afbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:30.083564 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31a9bd-5ac3-4719-8c23-2609f64afbb1 None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:30.084274 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c31a9bd-5ac3-4719-8c23-2609f64afbb1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:30.085745 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c31a9bd-5ac3-4719-8c23-2609f64afbb1 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:30.110986 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 380/738] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Jun 15 19:25:29 2022] POST /identity/v3/services => generated 251 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:25:30.110986 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c31a9bd-5ac3-4719-8c23-2609f64afbb1 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:30.179254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f2532e-b48e-4ac9-b23f-644cc71c12f5 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:30.229384 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2fbeec-c006-4ed0-9e54-ec26847a544c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:30.230249 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2fbeec-c006-4ed0-9e54-ec26847a544c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:30.230585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2fbeec-c006-4ed0-9e54-ec26847a544c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:30.230920 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2fbeec-c006-4ed0-9e54-ec26847a544c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:30.231576 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:30.244915 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2fbeec-c006-4ed0-9e54-ec26847a544c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:30.246362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2fbeec-c006-4ed0-9e54-ec26847a544c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:30.253929 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c31a9bd-5ac3-4719-8c23-2609f64afbb1 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:30.266718 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-bf2fbeec-c006-4ed0-9e54-ec26847a544c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:30.266718 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:30.266718 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:30.266718 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:30.266718 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:30.266718 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:30.266718 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:30.266718 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:30.266718 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:30.266718 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:30.266718 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:30.266718 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:30.266718 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:30.266718 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:30.266718 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:30.267682 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:30.267682 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:30.267682 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:30.267682 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:30.267682 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:30.267682 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:30.267682 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:30.267682 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:30.267682 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:30.267682 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:30.267682 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:30.267682 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:30.267682 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:30.267682 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:30.267682 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:30.267682 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:30.267682 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:30.268570 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:30.268570 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:30.268570 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:30.268570 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:30.272862 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 359/739] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:30.274121 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2fbeec-c006-4ed0-9e54-ec26847a544c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:30.278241 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:30.302595 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad870b5-770b-4ed0-90a9-599259fddb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:30.303883 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad870b5-770b-4ed0-90a9-599259fddb8b None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:30.304430 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad870b5-770b-4ed0-90a9-599259fddb8b None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:30.304875 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad870b5-770b-4ed0-90a9-599259fddb8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:30.321338 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad870b5-770b-4ed0-90a9-599259fddb8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:25:30.331820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-3ad870b5-770b-4ed0-90a9-599259fddb8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:25:30.331820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:30.331820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:30.331820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:30.331820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:30.331820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:30.331820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:30.331820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:30.331820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:30.331820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:30.331820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:30.331820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:30.331820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:25:30.331820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:25:30.331820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:25:30.331820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:25:30.332806 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:25:30.332806 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:25:30.332806 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:25:30.332806 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:25:30.332806 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:30.332806 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:30.332806 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:30.332806 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:30.332806 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:30.332806 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:30.332806 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:30.332806 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:30.332806 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:30.332806 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:30.332806 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:30.332806 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:30.334915 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:30.334915 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:30.334915 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:30.334915 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:30.334915 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:25:30.334915 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:25:30.334915 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:25:30.334915 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:25:30.334915 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:25:30.334915 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:25:30.334915 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:25:30.334915 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:30.334915 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 381/740] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jun 15 19:25:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:30.334915 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad870b5-770b-4ed0-90a9-599259fddb8b None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:30.404423 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2fbeec-c006-4ed0-9e54-ec26847a544c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:30.439288 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce2a12f-52b3-4e18-9fe2-d45984207020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:30.439960 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2a12f-52b3-4e18-9fe2-d45984207020 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:30.440227 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2a12f-52b3-4e18-9fe2-d45984207020 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:30.440500 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2a12f-52b3-4e18-9fe2-d45984207020 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:30.441222 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce2a12f-52b3-4e18-9fe2-d45984207020 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:30.441525 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce2a12f-52b3-4e18-9fe2-d45984207020 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:30.442920 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce2a12f-52b3-4e18-9fe2-d45984207020 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:30.446575 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad870b5-770b-4ed0-90a9-599259fddb8b None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:30.460122 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 360/741] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:30.460122 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce2a12f-52b3-4e18-9fe2-d45984207020 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:30.486899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532c4a74-ac39-4e03-879b-957a654f7c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:30.487575 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c4a74-ac39-4e03-879b-957a654f7c12 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:30.487861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c4a74-ac39-4e03-879b-957a654f7c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:30.488143 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c4a74-ac39-4e03-879b-957a654f7c12 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:30.488863 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532c4a74-ac39-4e03-879b-957a654f7c12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:30.489173 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532c4a74-ac39-4e03-879b-957a654f7c12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:30.490582 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532c4a74-ac39-4e03-879b-957a654f7c12 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:30.503279 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 382/742] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:30.504707 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa392386-c36a-4a77-90dc-2a8ecd0b8510 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:30.504938 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa392386-c36a-4a77-90dc-2a8ecd0b8510 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:30.505215 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa392386-c36a-4a77-90dc-2a8ecd0b8510 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:30.506638 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 383/743] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:30.507236 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa392386-c36a-4a77-90dc-2a8ecd0b8510 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:30.518215 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:30.531770 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:30.573093 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce2a12f-52b3-4e18-9fe2-d45984207020 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:30.610725 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fcbe48-4c92-4dc1-84b2-e2861139fb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:30.611394 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fcbe48-4c92-4dc1-84b2-e2861139fb46 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:30.611695 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fcbe48-4c92-4dc1-84b2-e2861139fb46 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:30.611946 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fcbe48-4c92-4dc1-84b2-e2861139fb46 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:30.612870 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fcbe48-4c92-4dc1-84b2-e2861139fb46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:30.612983 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fcbe48-4c92-4dc1-84b2-e2861139fb46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:30.614361 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fcbe48-4c92-4dc1-84b2-e2861139fb46 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:30.616759 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa392386-c36a-4a77-90dc-2a8ecd0b8510 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:30.627805 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 361/744] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:25:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:30.627805 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f15568-d86a-42de-bc09-ab33c770aaa6 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:30.627805 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f15568-d86a-42de-bc09-ab33c770aaa6 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:30.627805 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f15568-d86a-42de-bc09-ab33c770aaa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:30.644717 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-d3f15568-d86a-42de-bc09-ab33c770aaa6 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:30.646072 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d3f15568-d86a-42de-bc09-ab33c770aaa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:30.666112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50233cb-a746-4291-b2ec-d886e2858d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:30.666112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50233cb-a746-4291-b2ec-d886e2858d56 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:30.666112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50233cb-a746-4291-b2ec-d886e2858d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:30.666112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50233cb-a746-4291-b2ec-d886e2858d56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:30.667316 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50233cb-a746-4291-b2ec-d886e2858d56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:30.668770 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50233cb-a746-4291-b2ec-d886e2858d56 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:30.682827 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-e50233cb-a746-4291-b2ec-d886e2858d56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:30.682827 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:30.682827 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:30.682827 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:30.682827 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:30.682827 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:30.682827 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:30.682827 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:30.682827 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:30.682827 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:30.682827 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:30.682827 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:30.682827 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 19:25:30.682827 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 19:25:30.682827 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 19:25:30.683766 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 19:25:30.683766 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:30.683766 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:30.683766 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:30.683766 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:30.683766 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:30.683766 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:30.683766 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:30.683766 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:30.683766 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:30.683766 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:30.683766 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:30.683766 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:30.683766 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:30.683766 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:30.683766 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:30.683766 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:30.684691 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 19:25:30.684691 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 19:25:30.684691 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:25:30.684691 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:25:30.684691 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:25:30.684691 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:25:30.684691 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:30.684691 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:30.684691 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 384/745] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:25:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:30.685606 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50233cb-a746-4291-b2ec-d886e2858d56 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:30.730127 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 362/746] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:30 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 19:25:30.730127 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f15568-d86a-42de-bc09-ab33c770aaa6 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:30.767312 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:30.788723 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:30.803561 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50233cb-a746-4291-b2ec-d886e2858d56 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:30.834730 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f15568-d86a-42de-bc09-ab33c770aaa6 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:30.841422 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9789c-18f1-4a5e-b8e4-cdd4cdc321e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:30.842276 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9789c-18f1-4a5e-b8e4-cdd4cdc321e0 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:30.842576 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9789c-18f1-4a5e-b8e4-cdd4cdc321e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:30.842836 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9789c-18f1-4a5e-b8e4-cdd4cdc321e0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:30.852355 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea9789c-18f1-4a5e-b8e4-cdd4cdc321e0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:30.854380 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea9789c-18f1-4a5e-b8e4-cdd4cdc321e0 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:30.868845 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-eea9789c-18f1-4a5e-b8e4-cdd4cdc321e0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 19:25:30.868845 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:30.868845 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:30.868845 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:30.868845 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:30.868845 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:30.868845 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:30.868845 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:30.868845 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:30.868845 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:30.868845 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:30.868845 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:30.868845 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 19:25:30.868845 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 19:25:30.868845 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 19:25:30.868845 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 19:25:30.869896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:30.869896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:30.869896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:25:30.869896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:30.869896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:25:30.869896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:30.869896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:30.869896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:30.869896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:30.869896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:30.869896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:30.869896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:30.869896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:30.869896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:30.869896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:30.869896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:30.869896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:30.869896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:30.871524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:30.871524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:30.871524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 19:25:30.871524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 19:25:30.871524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 19:25:30.871524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 19:25:30.871524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 19:25:30.871524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 19:25:30.871524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 19:25:30.871524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:30.871524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 385/747] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Jun 15 19:25:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:30.871524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9789c-18f1-4a5e-b8e4-cdd4cdc321e0 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:30.877374 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5745c8-41a8-4249-a9e6-7bdd5ebfd047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:30.878216 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5745c8-41a8-4249-a9e6-7bdd5ebfd047 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:30.878498 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5745c8-41a8-4249-a9e6-7bdd5ebfd047 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:30.878770 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5745c8-41a8-4249-a9e6-7bdd5ebfd047 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:30.890446 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5745c8-41a8-4249-a9e6-7bdd5ebfd047 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:30.891888 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5745c8-41a8-4249-a9e6-7bdd5ebfd047 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:30.910288 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-9c5745c8-41a8-4249-a9e6-7bdd5ebfd047 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:30.910288 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:30.910288 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:30.910288 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:30.910288 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:30.910288 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:30.910288 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:30.910288 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:30.910288 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:30.910288 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:30.910288 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:30.910288 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:30.910288 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:30.910288 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:30.910288 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:30.911205 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:30.911205 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:30.911205 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:30.911205 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:30.911205 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:30.911205 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:30.911205 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:30.911205 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:30.911205 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:30.911205 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:30.911205 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:30.911205 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:30.911205 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:30.911205 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:30.911205 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:30.911205 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:30.911205 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:30.912112 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:30.912112 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:30.912112 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:30.912112 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:30.912112 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 363/748] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:30.912396 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5745c8-41a8-4249-a9e6-7bdd5ebfd047 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:30.980791 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9789c-18f1-4a5e-b8e4-cdd4cdc321e0 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:31.007669 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:31.018592 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd127da-3c2d-40ee-a1b0-adc7b683c555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:31.019179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd127da-3c2d-40ee-a1b0-adc7b683c555 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:31.019460 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd127da-3c2d-40ee-a1b0-adc7b683c555 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:31.019736 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd127da-3c2d-40ee-a1b0-adc7b683c555 None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:31.020512 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd127da-3c2d-40ee-a1b0-adc7b683c555 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:31.020819 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd127da-3c2d-40ee-a1b0-adc7b683c555 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:31.022011 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5745c8-41a8-4249-a9e6-7bdd5ebfd047 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:31.022285 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd127da-3c2d-40ee-a1b0-adc7b683c555 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:31.032383 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:31.036227 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 386/749] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jun 15 19:25:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:31.042065 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd127da-3c2d-40ee-a1b0-adc7b683c555 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:31.060772 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffffd40b-0030-4d09-9387-1f1ee26fb019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:31.061438 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffd40b-0030-4d09-9387-1f1ee26fb019 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:31.061719 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffd40b-0030-4d09-9387-1f1ee26fb019 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:31.062001 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffd40b-0030-4d09-9387-1f1ee26fb019 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:31.062748 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffffd40b-0030-4d09-9387-1f1ee26fb019 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:31.064127 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffffd40b-0030-4d09-9387-1f1ee26fb019 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:31.080974 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 364/750] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Jun 15 19:25:30 2022] POST /identity/v3/services => generated 240 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:25:31.082123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffffd40b-0030-4d09-9387-1f1ee26fb019 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:31.175011 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd127da-3c2d-40ee-a1b0-adc7b683c555 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:31.224748 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffffd40b-0030-4d09-9387-1f1ee26fb019 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:31.238953 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88088dd-1277-4aa9-8e6b-56976afc2fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:31.239668 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88088dd-1277-4aa9-8e6b-56976afc2fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:31.239955 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88088dd-1277-4aa9-8e6b-56976afc2fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:31.240230 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88088dd-1277-4aa9-8e6b-56976afc2fe7 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:31.241085 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88088dd-1277-4aa9-8e6b-56976afc2fe7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:31.241394 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88088dd-1277-4aa9-8e6b-56976afc2fe7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:31.243752 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88088dd-1277-4aa9-8e6b-56976afc2fe7 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:31.274845 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 387/751] 10.222.0.15 () {56 vars in 1149 bytes} [Wed Jun 15 19:25:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:31.274845 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88088dd-1277-4aa9-8e6b-56976afc2fe7 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:31.278550 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d3021-5c7d-43ed-8b88-c40449515668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:31.279448 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d3021-5c7d-43ed-8b88-c40449515668 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:31.279723 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d3021-5c7d-43ed-8b88-c40449515668 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:31.280002 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d3021-5c7d-43ed-8b88-c40449515668 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:31.280741 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d3021-5c7d-43ed-8b88-c40449515668 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:31.281104 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d3021-5c7d-43ed-8b88-c40449515668 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:31.282543 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d3021-5c7d-43ed-8b88-c40449515668 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:31.300086 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 365/752] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:31.301249 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d3021-5c7d-43ed-8b88-c40449515668 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:31.326603 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:31.336246 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:31.394398 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88088dd-1277-4aa9-8e6b-56976afc2fe7 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:31.435044 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d4b90-c976-4291-8642-c65674e2e76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:31.435675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d4b90-c976-4291-8642-c65674e2e76a None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:31.435955 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d4b90-c976-4291-8642-c65674e2e76a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:31.436230 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d4b90-c976-4291-8642-c65674e2e76a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:31.437060 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d4b90-c976-4291-8642-c65674e2e76a None admin] RBAC: Adding query filter params (role.id=4f736313c26d4f8db48bbb2f346e50e4, scope.project.id=7fb8cc0eb560465e972ffd45867ee73b, user.id=176d5775a70f4524a9b7fe1ce9d426ed) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:31.437373 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d4b90-c976-4291-8642-c65674e2e76a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:31.438886 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d4b90-c976-4291-8642-c65674e2e76a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:31.445039 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 388/753] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:25:31 2022] GET /identity/v3/role_assignments?user.id=176d5775a70f4524a9b7fe1ce9d426ed&scope.project.id=7fb8cc0eb560465e972ffd45867ee73b&role.id=4f736313c26d4f8db48bbb2f346e50e4 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:31.445357 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d4b90-c976-4291-8642-c65674e2e76a None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:31.450205 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d3021-5c7d-43ed-8b88-c40449515668 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:31.509282 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8228fd-ed0a-49f3-ae42-21f7045beaea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:31.510077 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8228fd-ed0a-49f3-ae42-21f7045beaea None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:31.510381 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8228fd-ed0a-49f3-ae42-21f7045beaea None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:31.510652 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8228fd-ed0a-49f3-ae42-21f7045beaea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:31.574277 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d4b90-c976-4291-8642-c65674e2e76a None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:31.577558 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:31.622990 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d36b605-53dc-4f03-98a2-c4e292fe0ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:31.625766 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d36b605-53dc-4f03-98a2-c4e292fe0ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:31.626520 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d36b605-53dc-4f03-98a2-c4e292fe0ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:31.627569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d36b605-53dc-4f03-98a2-c4e292fe0ae8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:31.639986 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d36b605-53dc-4f03-98a2-c4e292fe0ae8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:31.642401 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d36b605-53dc-4f03-98a2-c4e292fe0ae8 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:31.660239 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-0d36b605-53dc-4f03-98a2-c4e292fe0ae8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 19:25:31.660239 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:31.660239 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:31.660239 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:31.660239 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:31.660239 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:31.660239 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:31.660239 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:31.660239 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:31.660239 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:31.660239 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:31.660239 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:31.660239 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 19:25:31.660239 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 19:25:31.660239 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 19:25:31.660239 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 19:25:31.661269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:31.661269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:31.661269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:25:31.661269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:31.661269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:25:31.661269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:31.661269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:31.661269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:31.661269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:31.661269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:31.661269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:31.661269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:31.661269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:31.661269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:31.661269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:31.661269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:31.661269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:31.661269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:31.662249 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:31.662249 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:31.662249 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 19:25:31.662249 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 19:25:31.662249 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 19:25:31.662249 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 19:25:31.662249 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 19:25:31.662249 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 19:25:31.662249 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 19:25:31.662249 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:31.665214 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 389/754] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Jun 15 19:25:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:31.670378 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d36b605-53dc-4f03-98a2-c4e292fe0ae8 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:31.672424 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8228fd-ed0a-49f3-ae42-21f7045beaea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:31.673897 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8228fd-ed0a-49f3-ae42-21f7045beaea None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:31.725270 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:31.816609 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 366/755] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:25:31 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 19:25:31.825504 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8228fd-ed0a-49f3-ae42-21f7045beaea None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:31.854501 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d36b605-53dc-4f03-98a2-c4e292fe0ae8 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:31.923991 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a60e2d-71bd-435b-b665-726544cf1f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:31.927610 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a60e2d-71bd-435b-b665-726544cf1f01 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:31.928108 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a60e2d-71bd-435b-b665-726544cf1f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:31.928628 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a60e2d-71bd-435b-b665-726544cf1f01 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:31.929676 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a60e2d-71bd-435b-b665-726544cf1f01 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:31.930411 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a60e2d-71bd-435b-b665-726544cf1f01 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:31.933107 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a60e2d-71bd-435b-b665-726544cf1f01 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:31.968216 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 390/756] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Jun 15 19:25:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:25:31.973502 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a60e2d-71bd-435b-b665-726544cf1f01 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:32.008595 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:32.025398 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8228fd-ed0a-49f3-ae42-21f7045beaea None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:32.051063 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:32.064163 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc3c8f-31db-47f5-9713-7cc62387f17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:32.064965 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc3c8f-31db-47f5-9713-7cc62387f17e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:32.065246 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc3c8f-31db-47f5-9713-7cc62387f17e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:32.065522 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc3c8f-31db-47f5-9713-7cc62387f17e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:32.094281 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc3c8f-31db-47f5-9713-7cc62387f17e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:32.096273 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc3c8f-31db-47f5-9713-7cc62387f17e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:32.130113 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a60e2d-71bd-435b-b665-726544cf1f01 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:32.130575 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-efdc3c8f-31db-47f5-9713-7cc62387f17e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:32.130575 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:32.130575 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:32.130575 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:32.130575 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:32.130575 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:32.130575 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:32.130575 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:32.130575 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:32.130575 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:32.130575 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:32.130575 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:32.130575 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:32.130575 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:32.130575 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:32.131507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:32.131507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:32.131507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:32.131507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:32.131507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:32.131507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:32.131507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:32.131507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:32.131507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:32.131507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:32.131507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:32.131507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:32.131507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:32.131507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:32.131507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:32.131507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:32.131507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:32.132426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:32.132426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:32.132426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:32.132426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:32.135030 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 367/757] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:32.140738 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc3c8f-31db-47f5-9713-7cc62387f17e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:32.164443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af8925-a36d-4c5c-b1c7-825fa18fb532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:32.165237 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af8925-a36d-4c5c-b1c7-825fa18fb532 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:32.165787 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af8925-a36d-4c5c-b1c7-825fa18fb532 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:32.166065 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af8925-a36d-4c5c-b1c7-825fa18fb532 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:32.187658 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26af8925-a36d-4c5c-b1c7-825fa18fb532 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:32.190455 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26af8925-a36d-4c5c-b1c7-825fa18fb532 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:32.226070 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-26af8925-a36d-4c5c-b1c7-825fa18fb532 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:32.226070 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:32.226070 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:32.226070 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:32.226070 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:32.226070 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:32.226070 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:32.226070 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:32.226070 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:32.226070 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:32.226070 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:32.226070 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:32.226070 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:32.226070 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:32.226070 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:32.227054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:32.227054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:32.227054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:32.227054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:32.227054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:32.227054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:32.227054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:32.227054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:32.227054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:32.227054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:32.227054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:32.227054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:32.227054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:32.227054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:32.227054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:32.227054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:32.227054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:32.227952 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:32.227952 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:32.227952 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:32.227952 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:32.227952 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 391/758] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:32.233831 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af8925-a36d-4c5c-b1c7-825fa18fb532 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:32.355072 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc3c8f-31db-47f5-9713-7cc62387f17e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:32.379545 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:32.409061 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af8925-a36d-4c5c-b1c7-825fa18fb532 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:32.410606 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f4508e-9187-4d96-94ef-1184f8286ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:32.411379 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4508e-9187-4d96-94ef-1184f8286ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:32.411671 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4508e-9187-4d96-94ef-1184f8286ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:32.411947 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4508e-9187-4d96-94ef-1184f8286ec7 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:32.412676 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f4508e-9187-4d96-94ef-1184f8286ec7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:32.412986 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f4508e-9187-4d96-94ef-1184f8286ec7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:32.414401 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f4508e-9187-4d96-94ef-1184f8286ec7 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:32.428860 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 368/759] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:25:32.430320 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be69b3-26d0-4f2b-87f0-20d2cf061a67 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:32.430616 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be69b3-26d0-4f2b-87f0-20d2cf061a67 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:32.431494 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be69b3-26d0-4f2b-87f0-20d2cf061a67 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:32.434191 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 369/760] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:32.434960 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be69b3-26d0-4f2b-87f0-20d2cf061a67 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:32.449739 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32f74c-22a5-4260-ae77-261e1ac9059d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:32.450439 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32f74c-22a5-4260-ae77-261e1ac9059d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:32.450719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32f74c-22a5-4260-ae77-261e1ac9059d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:32.450993 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32f74c-22a5-4260-ae77-261e1ac9059d None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:32.451835 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da32f74c-22a5-4260-ae77-261e1ac9059d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:32.452153 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da32f74c-22a5-4260-ae77-261e1ac9059d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:32.453642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da32f74c-22a5-4260-ae77-261e1ac9059d None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:32.467532 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 392/761] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:32.468920 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3ed2a-b689-4d97-b136-46e5082e3bf5 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:32.469299 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3ed2a-b689-4d97-b136-46e5082e3bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:32.469578 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3ed2a-b689-4d97-b136-46e5082e3bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:32.486676 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:32.489359 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-08e3ed2a-b689-4d97-b136-46e5082e3bf5 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:32.490697 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-08e3ed2a-b689-4d97-b136-46e5082e3bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:32.571465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be69b3-26d0-4f2b-87f0-20d2cf061a67 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:32.610990 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03104567-bd7f-41a1-9b08-ef847143f7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:32.611841 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03104567-bd7f-41a1-9b08-ef847143f7cb None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:32.612198 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03104567-bd7f-41a1-9b08-ef847143f7cb None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:32.612541 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03104567-bd7f-41a1-9b08-ef847143f7cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:32.620653 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03104567-bd7f-41a1-9b08-ef847143f7cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:32.622342 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03104567-bd7f-41a1-9b08-ef847143f7cb None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:32.639049 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-03104567-bd7f-41a1-9b08-ef847143f7cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:32.639049 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:32.639049 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:32.639049 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:32.639049 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:32.639049 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:32.639049 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:32.639049 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:32.639049 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:32.639049 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:32.639049 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:32.639049 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:32.639049 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 19:25:32.639049 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 19:25:32.639049 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 19:25:32.641219 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 19:25:32.641219 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:32.641219 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:32.641219 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:32.641219 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:32.641219 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:32.641219 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:32.641219 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:32.641219 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:32.641219 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:32.641219 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:32.641219 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:32.641219 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:32.641219 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:32.641219 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:32.641219 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:32.641219 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:32.643484 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 19:25:32.643484 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 19:25:32.643484 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:25:32.643484 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:25:32.643484 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:25:32.643484 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:25:32.643484 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:32.643484 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:32.643484 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 370/762] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:25:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:32.643484 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03104567-bd7f-41a1-9b08-ef847143f7cb None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:32.678982 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 393/763] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:25:32.679929 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3ed2a-b689-4d97-b136-46e5082e3bf5 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:32.754450 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:32.767603 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:32.774792 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03104567-bd7f-41a1-9b08-ef847143f7cb None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:32.798732 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3ed2a-b689-4d97-b136-46e5082e3bf5 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:32.826724 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf6b930-db88-4735-bad8-9bab5a399220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:32.828513 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6b930-db88-4735-bad8-9bab5a399220 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:32.829254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6b930-db88-4735-bad8-9bab5a399220 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:32.829683 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6b930-db88-4735-bad8-9bab5a399220 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:32.839885 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a377f9-2995-48b4-89b4-4da467b41c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:32.840217 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf6b930-db88-4735-bad8-9bab5a399220 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:32.840763 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a377f9-2995-48b4-89b4-4da467b41c88 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:32.841141 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a377f9-2995-48b4-89b4-4da467b41c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:32.841511 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a377f9-2995-48b4-89b4-4da467b41c88 None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:32.842399 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf6b930-db88-4735-bad8-9bab5a399220 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:32.842532 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a377f9-2995-48b4-89b4-4da467b41c88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:32.842914 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a377f9-2995-48b4-89b4-4da467b41c88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:32.844438 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a377f9-2995-48b4-89b4-4da467b41c88 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:32.861645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 394/764] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jun 15 19:25:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:32.861802 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-ccf6b930-db88-4735-bad8-9bab5a399220 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:32.861802 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:32.861802 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:32.861802 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:32.861802 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:32.861802 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:32.861802 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:32.861802 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:32.861802 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:32.861802 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:32.861802 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:32.861802 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:32.861802 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 19:25:32.861802 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 19:25:32.861802 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 19:25:32.862683 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 19:25:32.862683 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:32.862683 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:32.862683 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:32.862683 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:32.862683 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:32.862683 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:32.862683 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:32.862683 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:32.862683 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:32.862683 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:32.862683 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:32.862683 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:32.862683 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:32.862683 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:32.862683 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:32.862683 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:32.863594 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 19:25:32.863594 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 19:25:32.863594 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:25:32.863594 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:25:32.863594 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:25:32.863594 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:25:32.863594 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:32.863594 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:32.865258 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 371/765] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:25:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:32.865949 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf6b930-db88-4735-bad8-9bab5a399220 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:32.866288 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a377f9-2995-48b4-89b4-4da467b41c88 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:32.980735 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a377f9-2995-48b4-89b4-4da467b41c88 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:32.997769 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf6b930-db88-4735-bad8-9bab5a399220 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:33.006621 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:33.019321 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d8a9a3-6e8c-40db-a19d-d166d923913b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:33.020484 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8a9a3-6e8c-40db-a19d-d166d923913b None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:33.020876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8a9a3-6e8c-40db-a19d-d166d923913b None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:33.021245 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8a9a3-6e8c-40db-a19d-d166d923913b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:33.022221 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d8a9a3-6e8c-40db-a19d-d166d923913b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:33.023730 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d8a9a3-6e8c-40db-a19d-d166d923913b None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:33.038780 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-20d8a9a3-6e8c-40db-a19d-d166d923913b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 19:25:33.038780 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:33.038780 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 19:25:33.038780 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 19:25:33.038780 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 19:25:33.038780 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 19:25:33.038780 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 19:25:33.038780 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:25:33.038780 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 19:25:33.038780 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:33.038780 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 19:25:33.038780 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:33.038780 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:33.038780 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:33.038780 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:33.038780 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:33.039959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:33.039959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:33.039959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:33.039959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:33.039959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:33.039959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:33.039959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:33.039959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 19:25:33.039959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 19:25:33.039959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 19:25:33.039959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 19:25:33.039959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:33.039959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:33.039959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:33.039959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:33.039959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:33.041675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:33.041675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:33.041675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:33.041675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:33.041675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:33.041675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:33.041675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:33.041675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:33.041675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:33.041675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:33.041675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:33.041675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 19:25:33.041675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:25:33.041675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 19:25:33.041675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:33.046222 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 395/766] 10.222.0.15 () {56 vars in 1129 bytes} [Wed Jun 15 19:25:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:33.047196 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d8a9a3-6e8c-40db-a19d-d166d923913b None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:33.048645 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d251c-c95f-4df6-9d68-39186de48aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:33.049511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d251c-c95f-4df6-9d68-39186de48aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:33.049797 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d251c-c95f-4df6-9d68-39186de48aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:33.050072 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d251c-c95f-4df6-9d68-39186de48aa8 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:33.050895 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284d251c-c95f-4df6-9d68-39186de48aa8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:33.051207 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284d251c-c95f-4df6-9d68-39186de48aa8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:33.052636 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284d251c-c95f-4df6-9d68-39186de48aa8 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:33.072119 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 372/767] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jun 15 19:25:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:33.073564 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d251c-c95f-4df6-9d68-39186de48aa8 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:33.101021 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:33.174343 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d8a9a3-6e8c-40db-a19d-d166d923913b None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:33.210488 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d251c-c95f-4df6-9d68-39186de48aa8 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:33.210954 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7893d9f-64d0-4139-9096-82a48f1c3a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:33.211846 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7893d9f-64d0-4139-9096-82a48f1c3a20 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:33.212351 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7893d9f-64d0-4139-9096-82a48f1c3a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:33.213389 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7893d9f-64d0-4139-9096-82a48f1c3a20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:33.214643 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7893d9f-64d0-4139-9096-82a48f1c3a20 None admin] RBAC: Adding query filter params (role.id=a8df9714563b435e9fce55a04e74ae5e, scope.project.id=7fb8cc0eb560465e972ffd45867ee73b, user.id=67aa048d409c46e9a729aec9d1b2286e) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:33.215172 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7893d9f-64d0-4139-9096-82a48f1c3a20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:33.216843 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7893d9f-64d0-4139-9096-82a48f1c3a20 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:33.230263 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 396/768] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:25:33 2022] GET /identity/v3/role_assignments?user.id=67aa048d409c46e9a729aec9d1b2286e&scope.project.id=7fb8cc0eb560465e972ffd45867ee73b&role.id=a8df9714563b435e9fce55a04e74ae5e => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:33.231575 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7893d9f-64d0-4139-9096-82a48f1c3a20 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:33.254126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f896122-a036-4dad-8377-33792c9c5e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:33.254126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f896122-a036-4dad-8377-33792c9c5e6c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:33.254126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f896122-a036-4dad-8377-33792c9c5e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:33.254126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f896122-a036-4dad-8377-33792c9c5e6c None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:33.256882 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f896122-a036-4dad-8377-33792c9c5e6c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:33.257210 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f896122-a036-4dad-8377-33792c9c5e6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:33.260601 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f896122-a036-4dad-8377-33792c9c5e6c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:33.272590 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 373/769] 10.222.0.15 () {56 vars in 1124 bytes} [Wed Jun 15 19:25:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:33.280004 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f896122-a036-4dad-8377-33792c9c5e6c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:33.321069 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:33.374809 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7893d9f-64d0-4139-9096-82a48f1c3a20 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:33.415806 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a9eb2-4e98-42bb-9b9b-932345db31a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:33.416723 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a9eb2-4e98-42bb-9b9b-932345db31a8 None admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:33.417006 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a9eb2-4e98-42bb-9b9b-932345db31a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:33.417285 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a9eb2-4e98-42bb-9b9b-932345db31a8 None admin] PATH_INFO: `/v3/projects/7fb8cc0eb560465e972ffd45867ee73b/users/3e74ac44e5814122a3226ac023f3688a/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:33.419700 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a9eb2-4e98-42bb-9b9b-932345db31a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=7fb8cc0eb560465e972ffd45867ee73b, user_id=3e74ac44e5814122a3226ac023f3688a, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:33.421178 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a9eb2-4e98-42bb-9b9b-932345db31a8 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:33.445408 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 397/770] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jun 15 19:25:33 2022] PUT /identity/v3/projects/7fb8cc0eb560465e972ffd45867ee73b/users/3e74ac44e5814122a3226ac023f3688a/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:25:33.459271 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:33.483916 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f896122-a036-4dad-8377-33792c9c5e6c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:33.531733 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f37f6-2d44-4317-93b2-a3967c5e30d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:33.532827 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f37f6-2d44-4317-93b2-a3967c5e30d4 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:33.533354 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f37f6-2d44-4317-93b2-a3967c5e30d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:33.533873 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f37f6-2d44-4317-93b2-a3967c5e30d4 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:33.534864 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f37f6-2d44-4317-93b2-a3967c5e30d4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:33.535368 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f37f6-2d44-4317-93b2-a3967c5e30d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:33.536896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de27237-6065-44b6-90aa-c5c1ddcf5e35 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:33.537190 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de27237-6065-44b6-90aa-c5c1ddcf5e35 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:33.537441 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f37f6-2d44-4317-93b2-a3967c5e30d4 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:33.537544 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de27237-6065-44b6-90aa-c5c1ddcf5e35 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:33.540849 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 398/771] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 19:25:33.544245 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02533fa-4628-4691-bfb2-c992d3cb2c90 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:33.544520 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02533fa-4628-4691-bfb2-c992d3cb2c90 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:33.544796 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02533fa-4628-4691-bfb2-c992d3cb2c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:33.547025 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 374/772] 10.222.0.15 () {56 vars in 1124 bytes} [Wed Jun 15 19:25:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:25:33.553163 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f37f6-2d44-4317-93b2-a3967c5e30d4 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:33.581158 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-d02533fa-4628-4691-bfb2-c992d3cb2c90 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:33.584617 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-d02533fa-4628-4691-bfb2-c992d3cb2c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:33.723395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 399/773] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 19:25:33.743178 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f37f6-2d44-4317-93b2-a3967c5e30d4 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:33.750508 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02533fa-4628-4691-bfb2-c992d3cb2c90 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:33.806931 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba3d8f-bb41-4544-9e5b-bb4ea05f1ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:33.808152 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba3d8f-bb41-4544-9e5b-bb4ea05f1ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:33.809434 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba3d8f-bb41-4544-9e5b-bb4ea05f1ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:33.809923 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba3d8f-bb41-4544-9e5b-bb4ea05f1ca1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:33.815547 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba3d8f-bb41-4544-9e5b-bb4ea05f1ca1 None admin] RBAC: Adding query filter params (interface=public, service_id=8d4ca92fef1149ed8ec842191f653b3f, region_id=RegionOne) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:33.816134 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba3d8f-bb41-4544-9e5b-bb4ea05f1ca1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:33.817762 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba3d8f-bb41-4544-9e5b-bb4ea05f1ca1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:33.825720 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:33.834835 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 375/774] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Jun 15 19:25:33 2022] GET /identity/v3/endpoints?service_id=8d4ca92fef1149ed8ec842191f653b3f&interface=public®ion_id=RegionOne => generated 194 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:33.864916 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba3d8f-bb41-4544-9e5b-bb4ea05f1ca1 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:33.888587 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:33.901072 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02533fa-4628-4691-bfb2-c992d3cb2c90 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:33.937878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15895b60-94cc-4028-867c-07a62bcdd392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:33.938721 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15895b60-94cc-4028-867c-07a62bcdd392 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:33.939007 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15895b60-94cc-4028-867c-07a62bcdd392 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:33.939283 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15895b60-94cc-4028-867c-07a62bcdd392 None admin] PATH_INFO: `/v3/services/network` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:33.939945 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15895b60-94cc-4028-867c-07a62bcdd392 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:33.941314 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15895b60-94cc-4028-867c-07a62bcdd392 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:33.953776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-15895b60-94cc-4028-867c-07a62bcdd392 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 19:25:33.953776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:33.953776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 19:25:33.953776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 19:25:33.953776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 19:25:33.953776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 19:25:33.953776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 19:25:33.953776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:25:33.953776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 19:25:33.953776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:33.953776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 19:25:33.953776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:33.953776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:33.953776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:33.953776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:33.953776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:33.953776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:33.954815 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:33.954815 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:33.954815 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:33.954815 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:33.954815 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:33.954815 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:33.954815 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 19:25:33.954815 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 19:25:33.954815 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 19:25:33.954815 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 19:25:33.954815 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:33.954815 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:33.954815 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:33.954815 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:33.954815 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:33.954815 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:33.955658 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:33.955658 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:33.955658 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:33.955658 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:33.955658 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:33.955658 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:33.955658 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:33.955658 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:33.955658 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:33.955658 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:33.955658 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 19:25:33.955658 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:25:33.955658 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 19:25:33.955658 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:33.955658 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 400/775] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:25:33 2022] GET /identity/v3/services/network => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:25:33.956983 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab000f45-7c95-4fd1-b75f-d518d5a67376 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:33.956983 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab000f45-7c95-4fd1-b75f-d518d5a67376 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:33.957247 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab000f45-7c95-4fd1-b75f-d518d5a67376 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:33.958752 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 401/776] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:33.962249 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab000f45-7c95-4fd1-b75f-d518d5a67376 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:34.017338 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba3d8f-bb41-4544-9e5b-bb4ea05f1ca1 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:34.054253 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4419d7a5-6383-4ce8-ab2d-57b9156711b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:34.054928 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419d7a5-6383-4ce8-ab2d-57b9156711b0 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:34.055214 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419d7a5-6383-4ce8-ab2d-57b9156711b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:34.055493 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419d7a5-6383-4ce8-ab2d-57b9156711b0 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:34.056816 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4419d7a5-6383-4ce8-ab2d-57b9156711b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:34.058549 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4419d7a5-6383-4ce8-ab2d-57b9156711b0 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:34.066634 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 376/777] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Jun 15 19:25:33 2022] GET /identity/v3/services => generated 772 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:34.067886 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d0148-fbd6-49d1-8cfd-a7a6dd2c2692 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:34.068180 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d0148-fbd6-49d1-8cfd-a7a6dd2c2692 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:34.068460 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d0148-fbd6-49d1-8cfd-a7a6dd2c2692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:34.078984 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab000f45-7c95-4fd1-b75f-d518d5a67376 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:34.088449 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-695d0148-fbd6-49d1-8cfd-a7a6dd2c2692 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:34.089671 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-695d0148-fbd6-49d1-8cfd-a7a6dd2c2692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:34.093225 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:34.173205 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 377/778] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:25:34.202333 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d0148-fbd6-49d1-8cfd-a7a6dd2c2692 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:34.205393 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab198ae-7f6f-4743-923f-d57f468713af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:34.207839 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab198ae-7f6f-4743-923f-d57f468713af None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:34.207839 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab198ae-7f6f-4743-923f-d57f468713af None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:34.207839 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab198ae-7f6f-4743-923f-d57f468713af None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:34.207839 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab198ae-7f6f-4743-923f-d57f468713af None admin] RBAC: Adding query filter params (name=network) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:34.208183 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:34.208259 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab198ae-7f6f-4743-923f-d57f468713af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:34.209257 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab198ae-7f6f-4743-923f-d57f468713af None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:34.220318 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 402/779] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Jun 15 19:25:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:25:34.221764 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab198ae-7f6f-4743-923f-d57f468713af None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:34.332513 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d0148-fbd6-49d1-8cfd-a7a6dd2c2692 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:34.372244 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b606c79-6dac-4463-8d56-d7ef315cd9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:34.373299 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b606c79-6dac-4463-8d56-d7ef315cd9d8 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:34.374019 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b606c79-6dac-4463-8d56-d7ef315cd9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:34.374471 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b606c79-6dac-4463-8d56-d7ef315cd9d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:34.389980 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b606c79-6dac-4463-8d56-d7ef315cd9d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:25:34.400583 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-4b606c79-6dac-4463-8d56-d7ef315cd9d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:25:34.400583 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:34.400583 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:34.400583 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:34.400583 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:34.400583 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:34.400583 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:34.400583 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:34.400583 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:34.400583 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:34.400583 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:34.400583 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:34.400583 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:25:34.400583 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:25:34.400583 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:25:34.400583 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:25:34.401820 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:25:34.401820 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:25:34.401820 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:25:34.401820 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:25:34.401820 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:34.401820 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:34.401820 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:34.401820 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:34.401820 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:34.401820 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:34.401820 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:34.401820 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:34.401820 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:34.401820 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:34.401820 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:34.401820 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:34.402899 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:34.402899 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:34.402899 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:34.402899 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:34.402899 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:25:34.402899 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:25:34.402899 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:25:34.402899 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:25:34.402899 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:25:34.402899 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:25:34.402899 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:25:34.402899 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:34.405788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 378/780] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jun 15 19:25:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:25:34.411206 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b606c79-6dac-4463-8d56-d7ef315cd9d8 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:34.430295 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab198ae-7f6f-4743-923f-d57f468713af None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:34.469076 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289b9322-ba3c-4211-8025-e42be831e47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:34.469935 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b9322-ba3c-4211-8025-e42be831e47f None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:34.470368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b9322-ba3c-4211-8025-e42be831e47f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:34.470741 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b9322-ba3c-4211-8025-e42be831e47f None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:34.471700 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289b9322-ba3c-4211-8025-e42be831e47f None admin] RBAC: Adding query filter params (type=network) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:34.472036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289b9322-ba3c-4211-8025-e42be831e47f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:34.473589 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289b9322-ba3c-4211-8025-e42be831e47f None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:34.480313 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 403/781] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Jun 15 19:25:34 2022] GET /identity/v3/services?type=network => generated 350 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:34.487286 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289b9322-ba3c-4211-8025-e42be831e47f None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:34.498426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:34.553534 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:34.560712 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b606c79-6dac-4463-8d56-d7ef315cd9d8 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:34.625680 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14da06f-3f6a-46cd-9d12-408d1c29e6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:34.626305 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14da06f-3f6a-46cd-9d12-408d1c29e6c5 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:34.626599 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14da06f-3f6a-46cd-9d12-408d1c29e6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:34.626877 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14da06f-3f6a-46cd-9d12-408d1c29e6c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:34.628788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14da06f-3f6a-46cd-9d12-408d1c29e6c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:34.629105 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14da06f-3f6a-46cd-9d12-408d1c29e6c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:34.630526 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14da06f-3f6a-46cd-9d12-408d1c29e6c5 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:34.649961 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 379/782] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:25:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:34.654446 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14da06f-3f6a-46cd-9d12-408d1c29e6c5 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:34.688188 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289b9322-ba3c-4211-8025-e42be831e47f None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:34.755337 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c36ba0-da5d-456a-819c-c045ad3dd857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:34.756111 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c36ba0-da5d-456a-819c-c045ad3dd857 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:34.756281 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c36ba0-da5d-456a-819c-c045ad3dd857 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:34.756554 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c36ba0-da5d-456a-819c-c045ad3dd857 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:34.757352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c36ba0-da5d-456a-819c-c045ad3dd857 None admin] RBAC: Adding query filter params (interface=public, service_id=42fbb2ee293249ccbe0dedcab662dce0, region_id=RegionOne) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:34.757662 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c36ba0-da5d-456a-819c-c045ad3dd857 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:34.761467 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c36ba0-da5d-456a-819c-c045ad3dd857 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:34.761707 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14da06f-3f6a-46cd-9d12-408d1c29e6c5 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:34.776267 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 404/783] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Jun 15 19:25:34 2022] GET /identity/v3/endpoints?service_id=42fbb2ee293249ccbe0dedcab662dce0&interface=public®ion_id=RegionOne => generated 194 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:34.781549 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c36ba0-da5d-456a-819c-c045ad3dd857 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:34.791404 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:34.806035 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c01a40-9e84-4001-8164-e6aa290c0d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:34.807099 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c01a40-9e84-4001-8164-e6aa290c0d3f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:34.807387 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c01a40-9e84-4001-8164-e6aa290c0d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:34.807662 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c01a40-9e84-4001-8164-e6aa290c0d3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:34.830930 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c01a40-9e84-4001-8164-e6aa290c0d3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:34.833722 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c01a40-9e84-4001-8164-e6aa290c0d3f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:34.862653 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-58c01a40-9e84-4001-8164-e6aa290c0d3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:34.862653 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:34.862653 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:34.862653 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:34.862653 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:34.862653 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:34.862653 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:34.862653 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:34.862653 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:34.862653 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:34.862653 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:34.862653 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:34.862653 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:34.862653 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:34.862653 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:34.863994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:34.863994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:34.863994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:34.863994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:34.863994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:34.863994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:34.863994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:34.863994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:34.863994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:34.863994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:34.863994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:34.863994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:34.863994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:34.863994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:34.863994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:34.863994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:34.863994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:34.864893 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:34.864893 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:34.864893 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:34.864893 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:34.864893 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 380/784] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:25:34.870057 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c01a40-9e84-4001-8164-e6aa290c0d3f None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:34.934025 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c36ba0-da5d-456a-819c-c045ad3dd857 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:34.946446 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:34.979633 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c01a40-9e84-4001-8164-e6aa290c0d3f None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:35.005474 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67aa347-70a1-4e70-bfa6-5240dd730991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:35.006171 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67aa347-70a1-4e70-bfa6-5240dd730991 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:35.006456 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67aa347-70a1-4e70-bfa6-5240dd730991 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:35.006732 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67aa347-70a1-4e70-bfa6-5240dd730991 None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:35.007421 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67aa347-70a1-4e70-bfa6-5240dd730991 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:35.009057 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67aa347-70a1-4e70-bfa6-5240dd730991 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:35.019621 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 405/785] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Jun 15 19:25:34 2022] GET /identity/v3/services => generated 772 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:25:35.027770 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2840cd1-dd22-42c3-8834-6c45bd38ceaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:35.028474 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2840cd1-dd22-42c3-8834-6c45bd38ceaf None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:35.028759 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2840cd1-dd22-42c3-8834-6c45bd38ceaf None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:35.029038 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2840cd1-dd22-42c3-8834-6c45bd38ceaf None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:35.029805 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2840cd1-dd22-42c3-8834-6c45bd38ceaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:35.030136 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2840cd1-dd22-42c3-8834-6c45bd38ceaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:35.031595 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2840cd1-dd22-42c3-8834-6c45bd38ceaf None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:35.056582 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 381/786] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:35.062242 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67aa347-70a1-4e70-bfa6-5240dd730991 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:35.221050 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67aa347-70a1-4e70-bfa6-5240dd730991 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:35.273297 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13be2c9d-66c4-4d20-b5d5-ea6c63ee68bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:35.274119 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be2c9d-66c4-4d20-b5d5-ea6c63ee68bd None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:35.274410 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be2c9d-66c4-4d20-b5d5-ea6c63ee68bd None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:35.274684 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be2c9d-66c4-4d20-b5d5-ea6c63ee68bd None admin] PATH_INFO: `/v3/users/swift` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:35.284964 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13be2c9d-66c4-4d20-b5d5-ea6c63ee68bd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:35.286449 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13be2c9d-66c4-4d20-b5d5-ea6c63ee68bd None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:35.301049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-13be2c9d-66c4-4d20-b5d5-ea6c63ee68bd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 19:25:35.301049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:35.301049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:35.301049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:35.301049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:35.301049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:35.301049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:35.301049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:35.301049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:35.301049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:35.301049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:35.301049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:35.301049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 19:25:35.301049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 19:25:35.301049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 19:25:35.301049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 19:25:35.301989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:35.301989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:35.301989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:25:35.301989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:35.301989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:25:35.301989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:35.301989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:35.301989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:35.301989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:35.301989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:35.301989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:35.301989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:35.301989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:35.301989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:35.301989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:35.301989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:35.301989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:35.301989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:35.302989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:35.302989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:35.302989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 19:25:35.302989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 19:25:35.302989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 19:25:35.302989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 19:25:35.302989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 19:25:35.302989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 19:25:35.302989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 19:25:35.302989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:35.306787 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 406/787] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jun 15 19:25:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:35.311170 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2840cd1-dd22-42c3-8834-6c45bd38ceaf None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:35.366333 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:35.435888 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2840cd1-dd22-42c3-8834-6c45bd38ceaf None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:35.481647 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec293cdc-ade9-4d53-9cd5-aa8f71477db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:35.482602 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec293cdc-ade9-4d53-9cd5-aa8f71477db8 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:35.483055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec293cdc-ade9-4d53-9cd5-aa8f71477db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:35.483481 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec293cdc-ade9-4d53-9cd5-aa8f71477db8 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:35.484483 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec293cdc-ade9-4d53-9cd5-aa8f71477db8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:35.484939 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec293cdc-ade9-4d53-9cd5-aa8f71477db8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:35.486553 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec293cdc-ade9-4d53-9cd5-aa8f71477db8 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:35.518116 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 382/788] 10.222.0.15 () {56 vars in 1135 bytes} [Wed Jun 15 19:25:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:35.518314 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13be2c9d-66c4-4d20-b5d5-ea6c63ee68bd None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:35.552276 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:35.629327 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13be2c9d-66c4-4d20-b5d5-ea6c63ee68bd None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:35.667995 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e3703-bbaa-4761-84d3-fa8e390e3f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:35.668795 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e3703-bbaa-4761-84d3-fa8e390e3f4e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:35.669076 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e3703-bbaa-4761-84d3-fa8e390e3f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:35.669350 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e3703-bbaa-4761-84d3-fa8e390e3f4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:35.679671 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963e3703-bbaa-4761-84d3-fa8e390e3f4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:35.681242 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963e3703-bbaa-4761-84d3-fa8e390e3f4e None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:35.700688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-963e3703-bbaa-4761-84d3-fa8e390e3f4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:35.700688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:35.700688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:35.700688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:35.700688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:35.700688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:35.700688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:35.700688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:35.700688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:35.700688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:35.700688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:35.700688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:35.700688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:35.700688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:35.700688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:35.701642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:35.701642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:35.701642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:35.701642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:35.701642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:35.701642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:35.701642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:35.701642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:35.701642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:35.701642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:35.701642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:35.701642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:35.701642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:35.701642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:35.701642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:35.701642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:35.701642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:35.702667 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:35.702667 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:35.702667 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:35.702667 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:35.702667 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 407/789] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:35.711277 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec293cdc-ade9-4d53-9cd5-aa8f71477db8 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:35.772653 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d0f56-92f2-438b-81b9-02f26cc2ad58 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:35.773170 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d0f56-92f2-438b-81b9-02f26cc2ad58 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:35.773248 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d0f56-92f2-438b-81b9-02f26cc2ad58 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:35.775789 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 408/790] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:35.783520 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9256494-566a-42a0-9483-f13d7acf153e None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:35.783942 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9256494-566a-42a0-9483-f13d7acf153e None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:35.784322 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9256494-566a-42a0-9483-f13d7acf153e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:35.818899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-a9256494-566a-42a0-9483-f13d7acf153e None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:35.820200 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-a9256494-566a-42a0-9483-f13d7acf153e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:35.838542 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:35.850343 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec293cdc-ade9-4d53-9cd5-aa8f71477db8 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:35.915133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686a8a8-b2d9-46c5-9958-a4acecc794ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:35.915953 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686a8a8-b2d9-46c5-9958-a4acecc794ff None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:35.916158 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686a8a8-b2d9-46c5-9958-a4acecc794ff None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:35.916435 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686a8a8-b2d9-46c5-9958-a4acecc794ff None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:35.917175 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d686a8a8-b2d9-46c5-9958-a4acecc794ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:35.917486 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d686a8a8-b2d9-46c5-9958-a4acecc794ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:35.918905 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d686a8a8-b2d9-46c5-9958-a4acecc794ff None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:35.941281 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 383/791] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:35.945642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 409/792] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:25:35.947235 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686a8a8-b2d9-46c5-9958-a4acecc794ff None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:35.990714 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9256494-566a-42a0-9483-f13d7acf153e None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:35.994242 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:36.108130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686a8a8-b2d9-46c5-9958-a4acecc794ff None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:36.133868 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9256494-566a-42a0-9483-f13d7acf153e None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:36.155145 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339346c-99ec-4955-ad9a-1691997a1177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:36.155947 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339346c-99ec-4955-ad9a-1691997a1177 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:36.156231 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339346c-99ec-4955-ad9a-1691997a1177 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:36.156507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339346c-99ec-4955-ad9a-1691997a1177 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:36.166633 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a339346c-99ec-4955-ad9a-1691997a1177 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:36.168919 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a339346c-99ec-4955-ad9a-1691997a1177 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:36.171634 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b9c97-37d1-4965-bb68-36a05099b66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:36.172529 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b9c97-37d1-4965-bb68-36a05099b66c None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:36.172906 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b9c97-37d1-4965-bb68-36a05099b66c None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:36.173273 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b9c97-37d1-4965-bb68-36a05099b66c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:36.183971 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155b9c97-37d1-4965-bb68-36a05099b66c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:25:36.186003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-a339346c-99ec-4955-ad9a-1691997a1177 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:36.186003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:36.186003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:36.186003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:36.186003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:36.186003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:36.186003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:36.186003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:36.186003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:36.186003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:36.186003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:36.186003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:36.186003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 19:25:36.186003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 19:25:36.186003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 19:25:36.187108 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 19:25:36.187108 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:36.187108 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:36.187108 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:36.187108 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:36.187108 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:36.187108 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:36.187108 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:36.187108 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:36.187108 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:36.187108 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:36.187108 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:36.187108 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:36.187108 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:36.187108 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:36.187108 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:36.187108 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:36.188426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 19:25:36.188426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 19:25:36.188426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:25:36.188426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:25:36.188426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:25:36.188426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:25:36.188426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:36.188426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:36.188426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 384/793] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:25:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:36.188910 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1e86f-3f12-4247-841b-ebf1756aff66 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:36.194131 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1e86f-3f12-4247-841b-ebf1756aff66 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:36.194131 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1e86f-3f12-4247-841b-ebf1756aff66 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:36.194749 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 385/794] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:36.195915 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23a112-ff37-488b-9f66-62fe156c2628 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:36.196200 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23a112-ff37-488b-9f66-62fe156c2628 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:36.196476 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23a112-ff37-488b-9f66-62fe156c2628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:36.200416 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-155b9c97-37d1-4965-bb68-36a05099b66c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:25:36.200416 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:36.200416 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:36.200416 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:36.200416 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:36.200416 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:36.200416 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:36.200416 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:36.200416 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:36.200416 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:36.200416 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:36.200416 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:36.200416 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:25:36.200416 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:25:36.200416 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:25:36.200416 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:25:36.202074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:25:36.202074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:25:36.202074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:25:36.202074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:25:36.202074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:36.202074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:36.202074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:36.202074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:36.202074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:36.202074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:36.202074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:36.202074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:36.202074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:36.202074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:36.202074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:36.202074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:36.204020 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:36.204020 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:36.204020 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:36.204020 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:36.204020 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:25:36.204020 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:25:36.204020 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:25:36.204020 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:25:36.204020 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:25:36.204020 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:25:36.204020 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:25:36.204020 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:36.204020 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 410/795] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jun 15 19:25:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:25:36.204020 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b9c97-37d1-4965-bb68-36a05099b66c None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:36.217676 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-5a23a112-ff37-488b-9f66-62fe156c2628 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:36.219350 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-5a23a112-ff37-488b-9f66-62fe156c2628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:36.232885 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:36.278734 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:36.304773 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 386/796] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 19:25:36.305624 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a23a112-ff37-488b-9f66-62fe156c2628 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:36.348971 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b9c97-37d1-4965-bb68-36a05099b66c None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:36.385547 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ac359-b843-4f29-9c93-feb7e6f8fc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:36.386509 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ac359-b843-4f29-9c93-feb7e6f8fc16 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:36.386992 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ac359-b843-4f29-9c93-feb7e6f8fc16 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:36.387452 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ac359-b843-4f29-9c93-feb7e6f8fc16 None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:36.388405 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6ac359-b843-4f29-9c93-feb7e6f8fc16 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:36.388887 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6ac359-b843-4f29-9c93-feb7e6f8fc16 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:36.390530 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6ac359-b843-4f29-9c93-feb7e6f8fc16 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:36.407743 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 411/797] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jun 15 19:25:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:36.408806 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ac359-b843-4f29-9c93-feb7e6f8fc16 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:36.438878 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a23a112-ff37-488b-9f66-62fe156c2628 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:36.480511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67754c83-5c27-4825-bfc8-38bc993986bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:36.481575 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67754c83-5c27-4825-bfc8-38bc993986bb None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:36.482060 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67754c83-5c27-4825-bfc8-38bc993986bb None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:36.482538 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67754c83-5c27-4825-bfc8-38bc993986bb None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:36.483480 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67754c83-5c27-4825-bfc8-38bc993986bb None admin] RBAC: Adding query filter params (name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:36.484058 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67754c83-5c27-4825-bfc8-38bc993986bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:36.485709 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67754c83-5c27-4825-bfc8-38bc993986bb None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:36.495491 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 387/798] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jun 15 19:25:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:36.496802 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67754c83-5c27-4825-bfc8-38bc993986bb None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:36.506171 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:36.536177 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ac359-b843-4f29-9c93-feb7e6f8fc16 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:36.563692 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:36.577855 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9d7c64-302c-4f06-9023-1ed883f76fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:36.579008 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d7c64-302c-4f06-9023-1ed883f76fbf None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:36.579525 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d7c64-302c-4f06-9023-1ed883f76fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:36.579997 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d7c64-302c-4f06-9023-1ed883f76fbf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:36.593483 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9d7c64-302c-4f06-9023-1ed883f76fbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:25:36.601861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-ae9d7c64-302c-4f06-9023-1ed883f76fbf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:25:36.601861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:36.601861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:36.601861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:36.601861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:36.601861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:36.601861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:36.601861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:36.601861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:36.601861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:36.601861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:36.601861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:36.601861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:25:36.601861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:25:36.601861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:25:36.601861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:25:36.602855 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:25:36.602855 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:25:36.602855 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:25:36.602855 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:25:36.602855 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:36.602855 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:36.602855 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:36.602855 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:36.602855 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:36.602855 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:36.602855 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:36.602855 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:36.602855 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:36.602855 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:36.602855 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:36.602855 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:36.603774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:36.603774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:36.603774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:36.603774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:36.603774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:25:36.603774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:25:36.603774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:25:36.603774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:25:36.603774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:25:36.603774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:25:36.603774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:25:36.603774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:36.606696 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 412/799] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jun 15 19:25:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:36.607731 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9d7c64-302c-4f06-9023-1ed883f76fbf None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:36.624098 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67754c83-5c27-4825-bfc8-38bc993986bb None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:36.661508 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c7a39e-baba-4b6f-b5cd-2921ab0c8506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:36.662891 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c7a39e-baba-4b6f-b5cd-2921ab0c8506 None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:36.663391 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c7a39e-baba-4b6f-b5cd-2921ab0c8506 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:36.663868 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c7a39e-baba-4b6f-b5cd-2921ab0c8506 None admin] PATH_INFO: `/v3/projects/7fb8cc0eb560465e972ffd45867ee73b/users/176d5775a70f4524a9b7fe1ce9d426ed/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:36.666902 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c7a39e-baba-4b6f-b5cd-2921ab0c8506 None admin] RBAC: Authorizing `identity:create_grant(project_id=7fb8cc0eb560465e972ffd45867ee73b, user_id=176d5775a70f4524a9b7fe1ce9d426ed, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:36.668845 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c7a39e-baba-4b6f-b5cd-2921ab0c8506 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:36.692742 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 388/800] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jun 15 19:25:36 2022] PUT /identity/v3/projects/7fb8cc0eb560465e972ffd45867ee73b/users/176d5775a70f4524a9b7fe1ce9d426ed/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:25:36.694118 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c7a39e-baba-4b6f-b5cd-2921ab0c8506 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:36.763138 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9d7c64-302c-4f06-9023-1ed883f76fbf None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:36.780362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:36.809600 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c84aaab-f865-4713-af35-5e16912653a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:36.810443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84aaab-f865-4713-af35-5e16912653a6 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:36.810777 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84aaab-f865-4713-af35-5e16912653a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:36.811648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84aaab-f865-4713-af35-5e16912653a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:36.824879 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c7a39e-baba-4b6f-b5cd-2921ab0c8506 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:36.826683 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c84aaab-f865-4713-af35-5e16912653a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:36.829133 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c84aaab-f865-4713-af35-5e16912653a6 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:36.853933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-2c84aaab-f865-4713-af35-5e16912653a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:36.853933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:36.853933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:36.853933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:36.853933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:36.853933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:36.853933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:36.853933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:36.853933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:36.853933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:36.853933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:36.853933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:36.853933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:36.853933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:36.853933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:36.854920 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:36.854920 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:36.854920 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:36.854920 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:36.854920 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:36.854920 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:36.854920 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:36.854920 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:36.854920 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:36.854920 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:36.854920 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:36.854920 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:36.854920 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:36.854920 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:36.854920 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:36.854920 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:36.854920 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:36.855820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:36.855820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:36.855820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:36.855820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:36.858340 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 413/801] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:36.859637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16a721-63b4-4f62-b4b3-62b899381c6c None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:36.859922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16a721-63b4-4f62-b4b3-62b899381c6c None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:36.860195 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16a721-63b4-4f62-b4b3-62b899381c6c None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:36.861610 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 414/802] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:36.864412 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef16a721-63b4-4f62-b4b3-62b899381c6c None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:36.879285 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f0c3a-d221-4706-b9b3-5882cf117cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:36.880008 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f0c3a-d221-4706-b9b3-5882cf117cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:36.880364 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f0c3a-d221-4706-b9b3-5882cf117cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:36.880902 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f0c3a-d221-4706-b9b3-5882cf117cb7 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:36.881872 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550f0c3a-d221-4706-b9b3-5882cf117cb7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:36.882351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550f0c3a-d221-4706-b9b3-5882cf117cb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:36.884702 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550f0c3a-d221-4706-b9b3-5882cf117cb7 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:36.895692 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 389/803] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:25:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:36.897533 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9af32-4c7f-4ca9-b0d8-c8136a3cf603 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:36.897816 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9af32-4c7f-4ca9-b0d8-c8136a3cf603 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:36.898115 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9af32-4c7f-4ca9-b0d8-c8136a3cf603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:36.909804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:36.921389 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c3c9af32-4c7f-4ca9-b0d8-c8136a3cf603 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:36.923436 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c3c9af32-4c7f-4ca9-b0d8-c8136a3cf603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:36.986905 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef16a721-63b4-4f62-b4b3-62b899381c6c None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:37.009693 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 390/804] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:25:37.010587 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c9af32-4c7f-4ca9-b0d8-c8136a3cf603 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:37.035328 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a125a79-6f5f-4c8f-a5b7-e67469873cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:37.036059 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a125a79-6f5f-4c8f-a5b7-e67469873cc5 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:37.036343 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a125a79-6f5f-4c8f-a5b7-e67469873cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:37.036670 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a125a79-6f5f-4c8f-a5b7-e67469873cc5 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:37.037399 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a125a79-6f5f-4c8f-a5b7-e67469873cc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:37.037704 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a125a79-6f5f-4c8f-a5b7-e67469873cc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:37.039219 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a125a79-6f5f-4c8f-a5b7-e67469873cc5 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:37.066515 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 415/805] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:25:37.067463 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a125a79-6f5f-4c8f-a5b7-e67469873cc5 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:37.072472 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:37.117775 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c9af32-4c7f-4ca9-b0d8-c8136a3cf603 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:37.158764 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac50af-fc08-456b-92e5-ea03084ad910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:37.159568 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac50af-fc08-456b-92e5-ea03084ad910 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:37.159849 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac50af-fc08-456b-92e5-ea03084ad910 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:37.160217 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac50af-fc08-456b-92e5-ea03084ad910 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:37.171632 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:37.172803 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bac50af-fc08-456b-92e5-ea03084ad910 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:37.174661 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bac50af-fc08-456b-92e5-ea03084ad910 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:37.178119 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a125a79-6f5f-4c8f-a5b7-e67469873cc5 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:37.194754 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-9bac50af-fc08-456b-92e5-ea03084ad910 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:37.194754 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:37.194754 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:37.194754 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:37.194754 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:37.194754 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:37.194754 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:37.194754 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:37.194754 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:37.194754 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:37.194754 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:37.194754 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:37.194754 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:37.194754 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:37.194754 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:37.195693 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:37.195693 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:37.195693 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:37.195693 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:37.195693 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:37.195693 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:37.195693 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:37.195693 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:37.195693 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:37.195693 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:37.195693 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:37.195693 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:37.195693 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:37.195693 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:37.195693 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:37.195693 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:37.195693 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:37.197065 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:37.197065 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:37.197065 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:37.197065 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:37.197065 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 391/806] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:37.197336 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac50af-fc08-456b-92e5-ea03084ad910 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:37.214902 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f1cd7c-f2ba-494a-aaed-fa8b50cf437f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:37.215925 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f1cd7c-f2ba-494a-aaed-fa8b50cf437f None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:37.216140 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f1cd7c-f2ba-494a-aaed-fa8b50cf437f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:37.216418 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f1cd7c-f2ba-494a-aaed-fa8b50cf437f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:37.225056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f1cd7c-f2ba-494a-aaed-fa8b50cf437f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:37.226537 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f1cd7c-f2ba-494a-aaed-fa8b50cf437f None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:37.243715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-d7f1cd7c-f2ba-494a-aaed-fa8b50cf437f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 19:25:37.243715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:37.243715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:37.243715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:37.243715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:37.243715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:37.243715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:37.243715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:37.243715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:37.243715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:37.243715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:37.243715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:37.243715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 19:25:37.243715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 19:25:37.243715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 19:25:37.243715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 19:25:37.244715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:37.244715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:37.244715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:25:37.244715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:37.244715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:25:37.244715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:37.244715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:37.244715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:37.244715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:37.244715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:37.244715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:37.244715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:37.244715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:37.244715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:37.244715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:37.244715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:37.244715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:37.244715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:37.245814 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:37.245814 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:37.245814 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 19:25:37.245814 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 19:25:37.245814 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 19:25:37.245814 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 19:25:37.245814 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 19:25:37.245814 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 19:25:37.245814 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 19:25:37.245814 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:37.249838 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 416/807] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Jun 15 19:25:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:37.250797 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f1cd7c-f2ba-494a-aaed-fa8b50cf437f None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:37.317311 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac50af-fc08-456b-92e5-ea03084ad910 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:37.343679 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:37.370208 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f1cd7c-f2ba-494a-aaed-fa8b50cf437f None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:37.371717 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814536ff-88d6-44a0-ba8b-db8782e622db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:37.372531 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814536ff-88d6-44a0-ba8b-db8782e622db None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:37.372861 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814536ff-88d6-44a0-ba8b-db8782e622db None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:37.373143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814536ff-88d6-44a0-ba8b-db8782e622db None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:37.374346 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814536ff-88d6-44a0-ba8b-db8782e622db None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:37.375203 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814536ff-88d6-44a0-ba8b-db8782e622db None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:37.387834 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-814536ff-88d6-44a0-ba8b-db8782e622db None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 19:25:37.387834 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:37.387834 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 19:25:37.387834 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 19:25:37.387834 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 19:25:37.387834 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 19:25:37.387834 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 19:25:37.387834 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:25:37.387834 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 19:25:37.387834 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:37.387834 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 19:25:37.387834 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:37.387834 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:37.387834 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:37.387834 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:37.387834 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:37.388784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:37.388784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:37.388784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:37.388784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:37.388784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:37.388784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:37.388784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:37.388784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 19:25:37.388784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 19:25:37.388784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 19:25:37.388784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 19:25:37.388784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:37.388784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:37.388784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:37.388784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:37.388784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:37.389620 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:37.389620 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:37.389620 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:37.389620 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:37.389620 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:37.389620 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:37.389620 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:37.389620 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:37.389620 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:37.389620 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:37.389620 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:37.389620 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 19:25:37.389620 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:25:37.389620 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 19:25:37.389620 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:37.394115 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 392/808] 10.222.0.15 () {56 vars in 1129 bytes} [Wed Jun 15 19:25:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:37.394936 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814536ff-88d6-44a0-ba8b-db8782e622db None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:37.413772 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ed52a-c842-4de6-89a7-9b998e6cbe0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:37.414466 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ed52a-c842-4de6-89a7-9b998e6cbe0e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:37.414746 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ed52a-c842-4de6-89a7-9b998e6cbe0e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:37.415016 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ed52a-c842-4de6-89a7-9b998e6cbe0e None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:37.416312 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ed52a-c842-4de6-89a7-9b998e6cbe0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:37.416615 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ed52a-c842-4de6-89a7-9b998e6cbe0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:37.417989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ed52a-c842-4de6-89a7-9b998e6cbe0e None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:37.438019 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 417/809] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:37.438924 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ed52a-c842-4de6-89a7-9b998e6cbe0e None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:37.457110 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:37.518737 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814536ff-88d6-44a0-ba8b-db8782e622db None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:37.542295 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ed52a-c842-4de6-89a7-9b998e6cbe0e None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:37.563918 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d6468-531e-4378-8045-5b0c7100b983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:37.565177 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d6468-531e-4378-8045-5b0c7100b983 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:37.565737 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d6468-531e-4378-8045-5b0c7100b983 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:37.566265 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d6468-531e-4378-8045-5b0c7100b983 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:37.567349 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d6468-531e-4378-8045-5b0c7100b983 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:37.567886 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d6468-531e-4378-8045-5b0c7100b983 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:37.569759 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d6468-531e-4378-8045-5b0c7100b983 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:37.580397 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a4e5c8-eb81-4b31-9eb8-f51e148d8469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:37.581018 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a4e5c8-eb81-4b31-9eb8-f51e148d8469 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:37.581299 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a4e5c8-eb81-4b31-9eb8-f51e148d8469 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:37.581573 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a4e5c8-eb81-4b31-9eb8-f51e148d8469 None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:37.582313 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a4e5c8-eb81-4b31-9eb8-f51e148d8469 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:37.583340 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a4e5c8-eb81-4b31-9eb8-f51e148d8469 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:37.584012 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a4e5c8-eb81-4b31-9eb8-f51e148d8469 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:37.593959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 418/810] 10.222.0.15 () {56 vars in 1124 bytes} [Wed Jun 15 19:25:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:37.593959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a4e5c8-eb81-4b31-9eb8-f51e148d8469 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:37.597891 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 393/811] 10.222.0.15 () {56 vars in 1149 bytes} [Wed Jun 15 19:25:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:37.598966 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d6468-531e-4378-8045-5b0c7100b983 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:37.636054 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:37.685616 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:37.707710 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a4e5c8-eb81-4b31-9eb8-f51e148d8469 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:37.710778 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d6468-531e-4378-8045-5b0c7100b983 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:37.747032 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c26493-5554-4696-a5da-e821de19e9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:37.747385 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b176635-38a3-4c13-9bbc-d5fc93358413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:37.747948 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c26493-5554-4696-a5da-e821de19e9a8 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:37.748172 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b176635-38a3-4c13-9bbc-d5fc93358413 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:37.748418 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c26493-5554-4696-a5da-e821de19e9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:37.748511 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b176635-38a3-4c13-9bbc-d5fc93358413 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:37.748713 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b176635-38a3-4c13-9bbc-d5fc93358413 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:37.748963 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c26493-5554-4696-a5da-e821de19e9a8 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:37.749767 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c26493-5554-4696-a5da-e821de19e9a8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:37.750201 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c26493-5554-4696-a5da-e821de19e9a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:37.752164 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c26493-5554-4696-a5da-e821de19e9a8 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:37.758668 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b176635-38a3-4c13-9bbc-d5fc93358413 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:37.758668 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b176635-38a3-4c13-9bbc-d5fc93358413 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:37.767364 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 394/812] 10.222.0.15 () {56 vars in 1124 bytes} [Wed Jun 15 19:25:37 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:37.767364 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c26493-5554-4696-a5da-e821de19e9a8 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:37.774818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-2b176635-38a3-4c13-9bbc-d5fc93358413 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 19:25:37.774818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:37.774818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:37.774818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:37.774818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:37.774818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:37.774818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:37.774818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:37.774818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:37.774818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:37.774818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:37.774818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:37.774818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 19:25:37.774818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 19:25:37.774818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 19:25:37.774818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 19:25:37.776448 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:37.776448 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:37.776448 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:25:37.776448 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:37.776448 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:25:37.776448 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:37.776448 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:37.776448 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:37.776448 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:37.776448 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:37.776448 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:37.776448 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:37.776448 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:37.776448 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:37.776448 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:37.776448 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:37.776448 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:37.776448 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:37.777398 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:37.777398 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:37.777398 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 19:25:37.777398 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 19:25:37.777398 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 19:25:37.777398 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 19:25:37.777398 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 19:25:37.777398 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 19:25:37.777398 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 19:25:37.777398 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:37.777398 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 419/813] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Jun 15 19:25:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:37.778359 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99e233-37b8-4f28-b725-bd626cb786da None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:37.778359 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99e233-37b8-4f28-b725-bd626cb786da None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:37.778657 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99e233-37b8-4f28-b725-bd626cb786da None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:37.784388 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 420/814] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:37.784815 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99e233-37b8-4f28-b725-bd626cb786da None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:37.872083 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:37.879280 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c26493-5554-4696-a5da-e821de19e9a8 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:37.894009 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99e233-37b8-4f28-b725-bd626cb786da None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:37.917452 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1fb9dc-55c8-4078-b774-6b3078e99619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:37.918281 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1fb9dc-55c8-4078-b774-6b3078e99619 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:37.918566 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1fb9dc-55c8-4078-b774-6b3078e99619 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:37.918848 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1fb9dc-55c8-4078-b774-6b3078e99619 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:37.929543 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1fb9dc-55c8-4078-b774-6b3078e99619 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:37.930111 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e197046-91f4-40f5-92eb-4a11862dbc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:37.930368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e197046-91f4-40f5-92eb-4a11862dbc75 None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:37.930651 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e197046-91f4-40f5-92eb-4a11862dbc75 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:37.930927 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e197046-91f4-40f5-92eb-4a11862dbc75 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:37.931082 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1fb9dc-55c8-4078-b774-6b3078e99619 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:37.931702 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e197046-91f4-40f5-92eb-4a11862dbc75 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:37.933048 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e197046-91f4-40f5-92eb-4a11862dbc75 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:37.944850 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:37.949629 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-7c1fb9dc-55c8-4078-b774-6b3078e99619 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:37.949629 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:37.949629 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:37.949629 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:37.949629 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:37.949629 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:37.949629 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:37.949629 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:37.949629 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:37.949629 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:37.949629 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:37.949629 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:37.949629 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:37.949629 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:37.949629 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:37.950815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:37.950815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:37.950815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:37.950815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:37.950815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:37.950815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:37.950815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:37.950815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:37.950815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:37.950815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:37.950815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:37.950815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:37.950815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:37.950815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:37.950815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:37.950815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:37.950815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:37.951699 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:37.951699 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:37.951699 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:37.951699 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:37.951699 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 395/815] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:37.956553 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f6966-71e3-4573-b7f8-ebd4aef934c0 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:37.956849 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f6966-71e3-4573-b7f8-ebd4aef934c0 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:37.957190 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f6966-71e3-4573-b7f8-ebd4aef934c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:37.959713 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 421/816] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Jun 15 19:25:37 2022] POST /identity/v3/endpoints => generated 346 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:25:37.959713 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e197046-91f4-40f5-92eb-4a11862dbc75 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:37.978152 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-6b0f6966-71e3-4573-b7f8-ebd4aef934c0 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:37.979425 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6b0f6966-71e3-4573-b7f8-ebd4aef934c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:38.072450 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 396/817] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:25:38.072450 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f6966-71e3-4573-b7f8-ebd4aef934c0 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:38.136878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e197046-91f4-40f5-92eb-4a11862dbc75 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:38.178541 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:38.185482 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d36b092-3dec-47c2-bb9b-b4379b17e9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:38.186378 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d36b092-3dec-47c2-bb9b-b4379b17e9a8 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:38.186490 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d36b092-3dec-47c2-bb9b-b4379b17e9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:38.186742 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d36b092-3dec-47c2-bb9b-b4379b17e9a8 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:38.187583 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d36b092-3dec-47c2-bb9b-b4379b17e9a8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:38.187890 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d36b092-3dec-47c2-bb9b-b4379b17e9a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:38.189327 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d36b092-3dec-47c2-bb9b-b4379b17e9a8 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:38.209630 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 422/818] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Jun 15 19:25:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:38.210548 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d36b092-3dec-47c2-bb9b-b4379b17e9a8 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:38.218891 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f6966-71e3-4573-b7f8-ebd4aef934c0 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:38.267763 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:38.271523 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af03f37-2ebe-4fdc-85f6-240dc0d088d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:38.273172 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af03f37-2ebe-4fdc-85f6-240dc0d088d1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:38.273467 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af03f37-2ebe-4fdc-85f6-240dc0d088d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:38.273745 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af03f37-2ebe-4fdc-85f6-240dc0d088d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:38.275491 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af03f37-2ebe-4fdc-85f6-240dc0d088d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:38.275882 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af03f37-2ebe-4fdc-85f6-240dc0d088d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:38.278711 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af03f37-2ebe-4fdc-85f6-240dc0d088d1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:38.296962 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 397/819] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:38.298256 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af03f37-2ebe-4fdc-85f6-240dc0d088d1 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:38.320181 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d36b092-3dec-47c2-bb9b-b4379b17e9a8 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:38.366361 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d960e-3410-45c7-b2c9-a2f19cf634e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:38.367407 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d960e-3410-45c7-b2c9-a2f19cf634e5 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:38.367847 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d960e-3410-45c7-b2c9-a2f19cf634e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:38.368366 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d960e-3410-45c7-b2c9-a2f19cf634e5 None admin] PATH_INFO: `/v3/services/network` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:38.369324 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65d960e-3410-45c7-b2c9-a2f19cf634e5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:38.370884 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65d960e-3410-45c7-b2c9-a2f19cf634e5 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:38.387275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-f65d960e-3410-45c7-b2c9-a2f19cf634e5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 19:25:38.387275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:38.387275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 19:25:38.387275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 19:25:38.387275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 19:25:38.387275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 19:25:38.387275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 19:25:38.387275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:25:38.387275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 19:25:38.387275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:38.387275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 19:25:38.387275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:38.387275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:38.387275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:38.387275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:38.387275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:38.387275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:38.388520 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:38.388520 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:38.388520 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:38.388520 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:38.388520 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:38.388520 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:38.388520 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 19:25:38.388520 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 19:25:38.388520 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 19:25:38.388520 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 19:25:38.388520 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:38.388520 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:38.388520 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:38.388520 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:38.388520 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:38.388520 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:38.389696 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:38.389696 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:38.389696 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:38.389696 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:38.389696 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:38.389696 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:38.389696 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:38.389696 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:38.389696 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:38.389696 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:38.389696 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 19:25:38.389696 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:25:38.389696 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 19:25:38.389696 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:38.392294 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 423/820] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:25:38 2022] GET /identity/v3/services/network => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:38.393304 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d960e-3410-45c7-b2c9-a2f19cf634e5 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:38.432452 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af03f37-2ebe-4fdc-85f6-240dc0d088d1 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:38.469589 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af42395-e22c-4f5d-bd5b-01b8bf970b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:38.470510 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af42395-e22c-4f5d-bd5b-01b8bf970b0a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:38.470781 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af42395-e22c-4f5d-bd5b-01b8bf970b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:38.471166 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af42395-e22c-4f5d-bd5b-01b8bf970b0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:38.473916 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:38.512017 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d960e-3410-45c7-b2c9-a2f19cf634e5 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:38.530739 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:38.548693 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb10855-a234-4206-bfcc-5125c57102b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:38.549818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb10855-a234-4206-bfcc-5125c57102b7 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:38.550333 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb10855-a234-4206-bfcc-5125c57102b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:38.550809 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb10855-a234-4206-bfcc-5125c57102b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:38.556751 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af42395-e22c-4f5d-bd5b-01b8bf970b0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:38.558251 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af42395-e22c-4f5d-bd5b-01b8bf970b0a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:38.558907 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb10855-a234-4206-bfcc-5125c57102b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:38.560536 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb10855-a234-4206-bfcc-5125c57102b7 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:38.576674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-cfb10855-a234-4206-bfcc-5125c57102b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:38.576674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:38.576674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:38.576674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:38.576674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:38.576674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:38.576674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:38.576674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:38.576674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:38.576674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:38.576674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:38.576674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:38.576674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 19:25:38.576674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 19:25:38.576674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 19:25:38.577638 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 19:25:38.577638 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:38.577638 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:38.577638 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:38.577638 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:38.577638 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:38.577638 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:38.577638 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:38.577638 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:38.577638 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:38.577638 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:38.577638 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:38.577638 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:38.577638 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:38.577638 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:38.577638 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:38.577638 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:38.578583 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 19:25:38.578583 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 19:25:38.578583 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:25:38.578583 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:25:38.578583 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:25:38.578583 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:25:38.578583 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:38.578583 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:38.581093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 424/821] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:25:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:38.582121 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb10855-a234-4206-bfcc-5125c57102b7 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:38.675764 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 398/822] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:25:38 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 19:25:38.675764 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af42395-e22c-4f5d-bd5b-01b8bf970b0a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:38.691418 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb10855-a234-4206-bfcc-5125c57102b7 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:38.722596 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:38.730904 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185249b-f17a-41a4-97a8-91c46801965d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:38.731732 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185249b-f17a-41a4-97a8-91c46801965d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:38.732121 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185249b-f17a-41a4-97a8-91c46801965d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:38.732500 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185249b-f17a-41a4-97a8-91c46801965d None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:38.733318 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b185249b-f17a-41a4-97a8-91c46801965d None admin] RBAC: Adding query filter params (name=network) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:38.733758 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b185249b-f17a-41a4-97a8-91c46801965d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:38.735300 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b185249b-f17a-41a4-97a8-91c46801965d None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:38.745747 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 425/823] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Jun 15 19:25:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:38.746798 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185249b-f17a-41a4-97a8-91c46801965d None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:38.797525 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af42395-e22c-4f5d-bd5b-01b8bf970b0a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:38.799056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:38.846688 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a28c20-8e7c-43e7-b591-dbdc05d86bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:38.847608 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a28c20-8e7c-43e7-b591-dbdc05d86bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:38.848603 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a28c20-8e7c-43e7-b591-dbdc05d86bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:38.849036 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a28c20-8e7c-43e7-b591-dbdc05d86bb6 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:38.850315 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a28c20-8e7c-43e7-b591-dbdc05d86bb6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:38.850777 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a28c20-8e7c-43e7-b591-dbdc05d86bb6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:38.853863 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a28c20-8e7c-43e7-b591-dbdc05d86bb6 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:38.861902 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185249b-f17a-41a4-97a8-91c46801965d None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:38.874628 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 399/824] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jun 15 19:25:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:38.874628 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a28c20-8e7c-43e7-b591-dbdc05d86bb6 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:38.914718 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8ca55e-037a-49e8-b5a6-28ddc8007100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:38.917906 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ca55e-037a-49e8-b5a6-28ddc8007100 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:38.918473 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ca55e-037a-49e8-b5a6-28ddc8007100 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:38.918959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ca55e-037a-49e8-b5a6-28ddc8007100 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:38.934715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8ca55e-037a-49e8-b5a6-28ddc8007100 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:38.937075 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8ca55e-037a-49e8-b5a6-28ddc8007100 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:38.961106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-7a8ca55e-037a-49e8-b5a6-28ddc8007100 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:38.961106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:38.961106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:38.961106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:38.961106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:38.961106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:38.961106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:38.961106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:38.961106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:38.961106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:38.961106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:38.961106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:38.961106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:38.961106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:38.961106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:38.962043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:38.962043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:38.962043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:38.962043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:38.962043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:38.962043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:38.962043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:38.962043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:38.962043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:38.962043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:38.962043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:38.962043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:38.962043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:38.962043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:38.962043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:38.962043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:38.962043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:38.963531 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:38.963531 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:38.963531 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:38.963531 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:38.966342 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 426/825] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:38.967373 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8ca55e-037a-49e8-b5a6-28ddc8007100 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:38.988050 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:38.996135 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a28c20-8e7c-43e7-b591-dbdc05d86bb6 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:39.032768 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bde171-b49c-4af2-9c5d-c563f7219fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:39.033435 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bde171-b49c-4af2-9c5d-c563f7219fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:39.033718 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bde171-b49c-4af2-9c5d-c563f7219fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:39.034000 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bde171-b49c-4af2-9c5d-c563f7219fb6 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:39.034746 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bde171-b49c-4af2-9c5d-c563f7219fb6 None admin] RBAC: Adding query filter params (type=network) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:39.035060 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bde171-b49c-4af2-9c5d-c563f7219fb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:39.036449 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bde171-b49c-4af2-9c5d-c563f7219fb6 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:39.042118 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 400/826] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Jun 15 19:25:38 2022] GET /identity/v3/services?type=network => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:39.043236 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32e505-9609-47e0-9213-b68c7c5d0a43 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:39.043525 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32e505-9609-47e0-9213-b68c7c5d0a43 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:39.043802 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32e505-9609-47e0-9213-b68c7c5d0a43 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:39.049968 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 401/827] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:39.049968 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f32e505-9609-47e0-9213-b68c7c5d0a43 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:39.079269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:39.090483 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8ca55e-037a-49e8-b5a6-28ddc8007100 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:39.134787 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e810450-466a-4a74-adf9-97f36ec3c0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:39.135933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e810450-466a-4a74-adf9-97f36ec3c0af None admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:39.136401 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e810450-466a-4a74-adf9-97f36ec3c0af None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:39.136819 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e810450-466a-4a74-adf9-97f36ec3c0af None admin] PATH_INFO: `/v3/projects/7fb8cc0eb560465e972ffd45867ee73b/users/67aa048d409c46e9a729aec9d1b2286e/roles/a8df9714563b435e9fce55a04e74ae5e` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:39.152896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e810450-466a-4a74-adf9-97f36ec3c0af None admin] RBAC: Authorizing `identity:create_grant(project_id=7fb8cc0eb560465e972ffd45867ee73b, user_id=67aa048d409c46e9a729aec9d1b2286e, role_id=a8df9714563b435e9fce55a04e74ae5e)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:39.154562 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e810450-466a-4a74-adf9-97f36ec3c0af None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:39.178188 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 427/828] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jun 15 19:25:38 2022] PUT /identity/v3/projects/7fb8cc0eb560465e972ffd45867ee73b/users/67aa048d409c46e9a729aec9d1b2286e/roles/a8df9714563b435e9fce55a04e74ae5e => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:25:39.178188 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e810450-466a-4a74-adf9-97f36ec3c0af None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:39.183026 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f32e505-9609-47e0-9213-b68c7c5d0a43 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:39.226447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32ca0c-8ed3-4e2f-bae5-5419e9590249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:39.227417 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32ca0c-8ed3-4e2f-bae5-5419e9590249 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:39.227924 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32ca0c-8ed3-4e2f-bae5-5419e9590249 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:39.228318 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32ca0c-8ed3-4e2f-bae5-5419e9590249 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:39.229222 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec32ca0c-8ed3-4e2f-bae5-5419e9590249 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:39.229686 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec32ca0c-8ed3-4e2f-bae5-5419e9590249 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:39.231531 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec32ca0c-8ed3-4e2f-bae5-5419e9590249 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:39.245312 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 402/829] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:39.247022 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16e707-fd50-456e-8bd1-0bd38eefa8a5 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:39.247489 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16e707-fd50-456e-8bd1-0bd38eefa8a5 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:39.247858 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16e707-fd50-456e-8bd1-0bd38eefa8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:39.266428 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-3f16e707-fd50-456e-8bd1-0bd38eefa8a5 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:39.267921 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3f16e707-fd50-456e-8bd1-0bd38eefa8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:39.283762 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:39.317789 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e810450-466a-4a74-adf9-97f36ec3c0af None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:39.359674 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 403/830] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:25:39.361438 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f16e707-fd50-456e-8bd1-0bd38eefa8a5 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:39.371675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a554e-b1fc-4d76-b821-25391ec43f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:39.372643 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a554e-b1fc-4d76-b821-25391ec43f60 None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:39.377170 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a554e-b1fc-4d76-b821-25391ec43f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:39.377774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a554e-b1fc-4d76-b821-25391ec43f60 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:39.378936 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9a554e-b1fc-4d76-b821-25391ec43f60 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:39.380709 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9a554e-b1fc-4d76-b821-25391ec43f60 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:39.399559 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:39.406869 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 428/831] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Jun 15 19:25:39 2022] POST /identity/v3/endpoints => generated 327 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:25:39.407680 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a554e-b1fc-4d76-b821-25391ec43f60 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:39.510404 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f16e707-fd50-456e-8bd1-0bd38eefa8a5 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:39.556709 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a554e-b1fc-4d76-b821-25391ec43f60 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:39.562554 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9d5d2-5c68-4f47-91cb-d1af69ca8232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:39.563618 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9d5d2-5c68-4f47-91cb-d1af69ca8232 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:39.564138 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9d5d2-5c68-4f47-91cb-d1af69ca8232 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:39.564604 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9d5d2-5c68-4f47-91cb-d1af69ca8232 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:39.572655 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da9d5d2-5c68-4f47-91cb-d1af69ca8232 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:39.574697 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da9d5d2-5c68-4f47-91cb-d1af69ca8232 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:39.591744 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-9da9d5d2-5c68-4f47-91cb-d1af69ca8232 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:39.591744 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:39.591744 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:39.591744 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:39.591744 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:39.591744 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:39.591744 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:39.591744 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:39.591744 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:39.591744 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:39.591744 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:39.591744 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:39.591744 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 19:25:39.591744 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 19:25:39.591744 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 19:25:39.592788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 19:25:39.592788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:39.592788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:39.592788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:39.592788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:39.592788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:39.592788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:39.592788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:39.592788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:39.592788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:39.592788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:39.592788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:39.592788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:39.592788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:39.592788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:39.592788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:39.592788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:39.593671 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 19:25:39.593671 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 19:25:39.593671 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:25:39.593671 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:25:39.593671 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:25:39.593671 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:25:39.593671 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:39.593671 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:39.599900 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 404/832] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:25:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:39.604011 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5dbcb5-8d08-43a7-b47e-5b4ef139c3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:39.604268 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9d5d2-5c68-4f47-91cb-d1af69ca8232 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:39.605375 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5dbcb5-8d08-43a7-b47e-5b4ef139c3bd None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:39.605861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5dbcb5-8d08-43a7-b47e-5b4ef139c3bd None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:39.606394 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5dbcb5-8d08-43a7-b47e-5b4ef139c3bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:39.609298 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:39.631477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5dbcb5-8d08-43a7-b47e-5b4ef139c3bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:25:39.640414 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-4e5dbcb5-8d08-43a7-b47e-5b4ef139c3bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:25:39.640414 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:39.640414 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:39.640414 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:39.640414 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:39.640414 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:39.640414 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:39.640414 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:39.640414 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:39.640414 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:39.640414 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:39.640414 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:39.640414 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:25:39.640414 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:25:39.640414 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:25:39.640414 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:25:39.641365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:25:39.641365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:25:39.641365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:25:39.641365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:25:39.641365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:39.641365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:39.641365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:39.641365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:39.641365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:39.641365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:39.641365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:39.641365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:39.641365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:39.641365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:39.641365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:39.641365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:39.642254 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:39.642254 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:39.642254 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:39.642254 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:39.642254 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:25:39.642254 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:25:39.642254 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:25:39.642254 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:25:39.642254 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:25:39.642254 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:25:39.642254 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:25:39.642254 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:39.643948 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 429/833] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jun 15 19:25:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:39.653257 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5dbcb5-8d08-43a7-b47e-5b4ef139c3bd None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:39.747059 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:39.754414 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9d5d2-5c68-4f47-91cb-d1af69ca8232 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:39.806924 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6895a49d-962e-4f36-bdb6-097180ec6db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:39.807523 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895a49d-962e-4f36-bdb6-097180ec6db2 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:39.808076 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895a49d-962e-4f36-bdb6-097180ec6db2 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:39.808363 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895a49d-962e-4f36-bdb6-097180ec6db2 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:39.809729 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6895a49d-962e-4f36-bdb6-097180ec6db2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:39.810066 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6895a49d-962e-4f36-bdb6-097180ec6db2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:39.812047 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6895a49d-962e-4f36-bdb6-097180ec6db2 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:39.816548 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5dbcb5-8d08-43a7-b47e-5b4ef139c3bd None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:39.829022 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 405/834] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jun 15 19:25:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:39.834776 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6895a49d-962e-4f36-bdb6-097180ec6db2 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:39.866716 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91db3a6-15b2-4816-aebb-6089d3e833c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:39.867715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91db3a6-15b2-4816-aebb-6089d3e833c8 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:39.868143 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91db3a6-15b2-4816-aebb-6089d3e833c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:39.868657 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91db3a6-15b2-4816-aebb-6089d3e833c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:39.870154 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91db3a6-15b2-4816-aebb-6089d3e833c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:39.870634 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91db3a6-15b2-4816-aebb-6089d3e833c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:39.872148 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91db3a6-15b2-4816-aebb-6089d3e833c8 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:39.881425 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 430/835] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:25:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:25:39.888565 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91db3a6-15b2-4816-aebb-6089d3e833c8 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:39.914886 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:39.968058 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6895a49d-962e-4f36-bdb6-097180ec6db2 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:40.000805 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91db3a6-15b2-4816-aebb-6089d3e833c8 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:40.015915 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a940c-2015-405a-bcab-aa046018d418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:40.017442 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a940c-2015-405a-bcab-aa046018d418 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:40.018233 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a940c-2015-405a-bcab-aa046018d418 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:40.019291 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:40.019548 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a940c-2015-405a-bcab-aa046018d418 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:40.020597 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4a940c-2015-405a-bcab-aa046018d418 None admin] RBAC: Adding query filter params (role.id=4f736313c26d4f8db48bbb2f346e50e4, scope.project.id=7fb8cc0eb560465e972ffd45867ee73b, user.id=3e74ac44e5814122a3226ac023f3688a) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:40.021121 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4a940c-2015-405a-bcab-aa046018d418 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:40.024327 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4a940c-2015-405a-bcab-aa046018d418 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:40.033789 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 406/836] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:25:39 2022] GET /identity/v3/role_assignments?user.id=3e74ac44e5814122a3226ac023f3688a&scope.project.id=7fb8cc0eb560465e972ffd45867ee73b&role.id=4f736313c26d4f8db48bbb2f346e50e4 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:25:40.048999 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627d622-f98b-43d5-a1bf-d2e1eb7c9c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:40.049877 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627d622-f98b-43d5-a1bf-d2e1eb7c9c04 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:40.050209 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627d622-f98b-43d5-a1bf-d2e1eb7c9c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:40.050533 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627d622-f98b-43d5-a1bf-d2e1eb7c9c04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:40.064067 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4627d622-f98b-43d5-a1bf-d2e1eb7c9c04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:40.065515 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4627d622-f98b-43d5-a1bf-d2e1eb7c9c04 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:40.085889 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-4627d622-f98b-43d5-a1bf-d2e1eb7c9c04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:40.085889 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:40.085889 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:40.085889 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:40.085889 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:40.085889 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:40.085889 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:40.085889 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:40.085889 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:40.085889 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:40.085889 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:40.085889 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:40.085889 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:40.085889 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:40.085889 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:40.086876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:40.086876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:40.086876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:40.086876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:40.086876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:40.086876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:40.086876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:40.086876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:40.086876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:40.086876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:40.086876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:40.086876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:40.086876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:40.086876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:40.086876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:40.086876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:40.086876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:40.087834 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:40.087834 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:40.087834 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:40.087834 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:40.087834 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 431/837] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:25:40.093368 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a940c-2015-405a-bcab-aa046018d418 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:40.213183 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a940c-2015-405a-bcab-aa046018d418 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:40.247861 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:40.256722 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae30c54-3dfd-4121-8fbd-01e1ddd2c4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:40.257386 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae30c54-3dfd-4121-8fbd-01e1ddd2c4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:40.257669 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae30c54-3dfd-4121-8fbd-01e1ddd2c4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:40.258221 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae30c54-3dfd-4121-8fbd-01e1ddd2c4f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:40.258953 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae30c54-3dfd-4121-8fbd-01e1ddd2c4f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:40.259809 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae30c54-3dfd-4121-8fbd-01e1ddd2c4f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:40.261190 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae30c54-3dfd-4121-8fbd-01e1ddd2c4f2 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:40.273435 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 407/838] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:40.278871 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627d622-f98b-43d5-a1bf-d2e1eb7c9c04 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:40.280554 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ccb3b-32f4-481e-88bf-d9a604e97307 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:40.280851 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ccb3b-32f4-481e-88bf-d9a604e97307 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:40.281128 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ccb3b-32f4-481e-88bf-d9a604e97307 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:40.283920 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 408/839] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:40.287948 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c41bb-8787-4f7b-ab99-f7b032098aff None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:40.288448 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c41bb-8787-4f7b-ab99-f7b032098aff None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:40.288930 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c41bb-8787-4f7b-ab99-f7b032098aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:40.307235 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c25c41bb-8787-4f7b-ab99-f7b032098aff None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:40.308424 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c25c41bb-8787-4f7b-ab99-f7b032098aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:40.398128 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627d622-f98b-43d5-a1bf-d2e1eb7c9c04 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:40.400249 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 409/840] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 19:25:40.422918 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c41bb-8787-4f7b-ab99-f7b032098aff None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:40.441688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729ada3-902d-4996-bcb7-831c918a2e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:40.442847 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729ada3-902d-4996-bcb7-831c918a2e52 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:40.443338 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729ada3-902d-4996-bcb7-831c918a2e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:40.443831 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729ada3-902d-4996-bcb7-831c918a2e52 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:40.452755 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d729ada3-902d-4996-bcb7-831c918a2e52 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:40.454531 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d729ada3-902d-4996-bcb7-831c918a2e52 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:40.470683 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-d729ada3-902d-4996-bcb7-831c918a2e52 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 19:25:40.470683 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:40.470683 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:40.470683 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:40.470683 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:40.470683 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:40.470683 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:40.470683 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:40.470683 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:40.470683 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:40.470683 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:40.470683 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:40.470683 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 19:25:40.470683 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 19:25:40.470683 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 19:25:40.470683 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 19:25:40.471687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:40.471687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:40.471687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:25:40.471687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:40.471687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:25:40.471687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:40.471687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:40.471687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:40.471687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:40.471687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:40.471687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:40.471687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:40.471687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:40.471687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:40.471687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:40.471687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:40.471687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:40.471687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:40.472645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:40.472645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:40.472645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 19:25:40.472645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 19:25:40.472645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 19:25:40.472645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 19:25:40.472645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 19:25:40.472645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 19:25:40.472645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 19:25:40.472645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:40.475519 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 432/841] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jun 15 19:25:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:40.482799 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729ada3-902d-4996-bcb7-831c918a2e52 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:40.526656 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:40.545002 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c41bb-8787-4f7b-ab99-f7b032098aff None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:40.570552 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:40.591153 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95471aa7-d4dd-4be7-8e4f-74adc88dfc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:40.592189 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95471aa7-d4dd-4be7-8e4f-74adc88dfc59 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:40.592314 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95471aa7-d4dd-4be7-8e4f-74adc88dfc59 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:40.592525 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95471aa7-d4dd-4be7-8e4f-74adc88dfc59 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:40.593237 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95471aa7-d4dd-4be7-8e4f-74adc88dfc59 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:40.594652 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95471aa7-d4dd-4be7-8e4f-74adc88dfc59 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:40.607010 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-95471aa7-d4dd-4be7-8e4f-74adc88dfc59 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 19:25:40.607010 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:40.607010 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 19:25:40.607010 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 19:25:40.607010 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 19:25:40.607010 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 19:25:40.607010 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 19:25:40.607010 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:25:40.607010 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 19:25:40.607010 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:40.607010 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 19:25:40.607010 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:40.607010 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:40.607010 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:40.607010 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:40.607010 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:40.607010 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:40.608023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:40.608023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:40.608023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:40.608023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:40.608023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:40.608023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:40.608023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 19:25:40.608023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 19:25:40.608023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 19:25:40.608023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 19:25:40.608023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:40.608023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:40.608023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:40.608023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:40.608023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:40.608023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:40.608848 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:40.608848 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:40.608848 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:40.608848 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:40.608848 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:40.608848 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:40.608848 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:40.608848 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:40.608848 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:40.608848 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:40.608848 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 19:25:40.608848 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:25:40.608848 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 19:25:40.608848 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:40.608848 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 410/842] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jun 15 19:25:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:25:40.614182 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95471aa7-d4dd-4be7-8e4f-74adc88dfc59 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:40.616749 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729ada3-902d-4996-bcb7-831c918a2e52 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:40.661749 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771dd15-646a-4213-b225-680c46697f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:40.662851 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771dd15-646a-4213-b225-680c46697f53 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:40.663139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771dd15-646a-4213-b225-680c46697f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:40.663414 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771dd15-646a-4213-b225-680c46697f53 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:40.664257 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c771dd15-646a-4213-b225-680c46697f53 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:40.664567 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c771dd15-646a-4213-b225-680c46697f53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:40.666574 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c771dd15-646a-4213-b225-680c46697f53 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:40.691898 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 433/843] 10.222.0.15 () {56 vars in 1135 bytes} [Wed Jun 15 19:25:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:25:40.692898 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771dd15-646a-4213-b225-680c46697f53 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:40.736309 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95471aa7-d4dd-4be7-8e4f-74adc88dfc59 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:40.781573 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a6f57-a4b5-422f-9e1d-15ddb18c1ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:40.782279 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a6f57-a4b5-422f-9e1d-15ddb18c1ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:40.782563 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a6f57-a4b5-422f-9e1d-15ddb18c1ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:40.782838 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a6f57-a4b5-422f-9e1d-15ddb18c1ef1 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:40.783567 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a6f57-a4b5-422f-9e1d-15ddb18c1ef1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:40.783869 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a6f57-a4b5-422f-9e1d-15ddb18c1ef1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:40.785232 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a6f57-a4b5-422f-9e1d-15ddb18c1ef1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:40.790788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 411/844] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jun 15 19:25:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:25:40.795804 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a6f57-a4b5-422f-9e1d-15ddb18c1ef1 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:40.809679 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:40.825219 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771dd15-646a-4213-b225-680c46697f53 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:40.850128 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:40.870545 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f0b77-afe4-4e3e-ac33-ed732ad5804d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:40.871100 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f0b77-afe4-4e3e-ac33-ed732ad5804d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:40.871444 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f0b77-afe4-4e3e-ac33-ed732ad5804d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:40.871870 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f0b77-afe4-4e3e-ac33-ed732ad5804d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:40.912686 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a6f57-a4b5-422f-9e1d-15ddb18c1ef1 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:40.941244 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073f0b77-afe4-4e3e-ac33-ed732ad5804d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:40.942837 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073f0b77-afe4-4e3e-ac33-ed732ad5804d None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:40.951759 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9997f2-680f-4357-b8ae-2a8b3c8ad0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:40.952699 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9997f2-680f-4357-b8ae-2a8b3c8ad0f6 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:40.953267 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9997f2-680f-4357-b8ae-2a8b3c8ad0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:40.953753 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9997f2-680f-4357-b8ae-2a8b3c8ad0f6 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:40.954800 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9997f2-680f-4357-b8ae-2a8b3c8ad0f6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:40.955348 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9997f2-680f-4357-b8ae-2a8b3c8ad0f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:40.956943 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9997f2-680f-4357-b8ae-2a8b3c8ad0f6 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:40.963184 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 412/845] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jun 15 19:25:40 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:41.045097 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:41.060963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 434/846] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:25:40 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 19:25:41.067257 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9997f2-680f-4357-b8ae-2a8b3c8ad0f6 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:41.181079 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9997f2-680f-4357-b8ae-2a8b3c8ad0f6 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:41.191764 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:41.217142 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7147ec-81f3-4e48-a607-c9e8626e9a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:41.217941 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7147ec-81f3-4e48-a607-c9e8626e9a2f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:41.218254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7147ec-81f3-4e48-a607-c9e8626e9a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:41.218535 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7147ec-81f3-4e48-a607-c9e8626e9a2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:41.229591 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7147ec-81f3-4e48-a607-c9e8626e9a2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:41.231123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7147ec-81f3-4e48-a607-c9e8626e9a2f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:41.248921 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-ab7147ec-81f3-4e48-a607-c9e8626e9a2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:41.248921 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:41.248921 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:41.248921 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:41.248921 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:41.248921 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:41.248921 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:41.248921 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:41.248921 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:41.248921 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:41.248921 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:41.248921 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:41.248921 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:41.248921 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:41.248921 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:41.250042 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:41.250042 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:41.250042 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:41.250042 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:41.250042 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:41.250042 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:41.250042 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:41.250042 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:41.250042 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:41.250042 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:41.250042 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:41.250042 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:41.250042 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:41.250042 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:41.250042 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:41.250042 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:41.250042 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:41.251020 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:41.251020 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:41.251020 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:41.251020 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:41.251020 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 413/847] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:41.255700 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f0b77-afe4-4e3e-ac33-ed732ad5804d None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:41.363315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f0b77-afe4-4e3e-ac33-ed732ad5804d None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:41.398737 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f425550d-ce0f-4457-8fde-988738517da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:41.399408 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f425550d-ce0f-4457-8fde-988738517da0 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:41.399751 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f425550d-ce0f-4457-8fde-988738517da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:41.400032 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f425550d-ce0f-4457-8fde-988738517da0 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:41.400752 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f425550d-ce0f-4457-8fde-988738517da0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:41.401063 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f425550d-ce0f-4457-8fde-988738517da0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:41.402454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f425550d-ce0f-4457-8fde-988738517da0 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:41.420359 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 435/848] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:41.425852 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7147ec-81f3-4e48-a607-c9e8626e9a2f None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:41.563849 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2f911-abd8-4c94-b1d1-00bcd69aa835 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:41.564226 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2f911-abd8-4c94-b1d1-00bcd69aa835 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:41.564422 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2f911-abd8-4c94-b1d1-00bcd69aa835 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:41.566050 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 436/849] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 19:25:41.572029 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c923b-c55b-48ef-9250-4f8592d52a5e None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:41.572322 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c923b-c55b-48ef-9250-4f8592d52a5e None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:41.572591 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c923b-c55b-48ef-9250-4f8592d52a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:41.590734 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-723c923b-c55b-48ef-9250-4f8592d52a5e None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:41.591951 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-723c923b-c55b-48ef-9250-4f8592d52a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:41.644663 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:41.654700 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7147ec-81f3-4e48-a607-c9e8626e9a2f None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:41.678236 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 437/850] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 19:25:41.702405 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d8db8-8be2-434d-bc46-4e0ae86e7fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:41.703549 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d8db8-8be2-434d-bc46-4e0ae86e7fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:41.704172 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d8db8-8be2-434d-bc46-4e0ae86e7fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:41.704646 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d8db8-8be2-434d-bc46-4e0ae86e7fe5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:41.705054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c923b-c55b-48ef-9250-4f8592d52a5e None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:41.713963 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092d8db8-8be2-434d-bc46-4e0ae86e7fe5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:41.716232 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092d8db8-8be2-434d-bc46-4e0ae86e7fe5 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:41.733440 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-092d8db8-8be2-434d-bc46-4e0ae86e7fe5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:41.733440 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:41.733440 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:41.733440 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:41.733440 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:41.733440 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:41.733440 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:41.733440 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:41.733440 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:41.733440 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:41.733440 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:41.733440 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:41.733440 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 19:25:41.733440 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 19:25:41.733440 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 19:25:41.734416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 19:25:41.734416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:41.734416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:41.734416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:41.734416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:41.734416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:41.734416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:41.734416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:41.734416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:41.734416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:41.734416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:41.734416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:41.734416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:41.734416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:41.734416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:41.734416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:41.734416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:41.735320 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 19:25:41.735320 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 19:25:41.735320 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:25:41.735320 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:25:41.735320 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:25:41.735320 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:25:41.735320 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:41.735320 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:41.735320 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 414/851] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:25:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:41.743804 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d8db8-8be2-434d-bc46-4e0ae86e7fe5 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:41.779353 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:41.849680 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c923b-c55b-48ef-9250-4f8592d52a5e None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:41.873767 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d8db8-8be2-434d-bc46-4e0ae86e7fe5 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:41.895247 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7b5b50-4010-499b-a502-7a2ffcd2edab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:41.896496 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b5b50-4010-499b-a502-7a2ffcd2edab None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:41.897025 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b5b50-4010-499b-a502-7a2ffcd2edab None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:41.897503 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b5b50-4010-499b-a502-7a2ffcd2edab None admin] PATH_INFO: `/v3/roles/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:41.909598 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7b5b50-4010-499b-a502-7a2ffcd2edab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:25:41.910108 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059bd59-1708-4808-a0c7-a93435037470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:41.910783 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059bd59-1708-4808-a0c7-a93435037470 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:41.911064 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059bd59-1708-4808-a0c7-a93435037470 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:41.911388 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059bd59-1708-4808-a0c7-a93435037470 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:41.912178 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059bd59-1708-4808-a0c7-a93435037470 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:41.912484 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059bd59-1708-4808-a0c7-a93435037470 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:41.913897 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059bd59-1708-4808-a0c7-a93435037470 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:41.919776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-9c7b5b50-4010-499b-a502-7a2ffcd2edab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:25:41.919776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:41.919776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:41.919776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:41.919776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:41.919776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:41.919776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:41.919776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:41.919776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:41.919776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:41.919776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:41.919776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:41.919776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:25:41.919776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:25:41.919776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:25:41.919776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:25:41.920705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:25:41.920705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:25:41.920705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:25:41.920705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:25:41.920705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:41.920705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:41.920705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:41.920705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:41.920705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:41.920705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:41.920705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:41.920705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:41.920705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:41.920705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:41.920705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:41.920705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:41.921552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:41.921552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:41.921552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:41.921552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:41.921552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:25:41.921552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:25:41.921552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:25:41.921552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:25:41.921552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:25:41.921552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:25:41.921552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:25:41.921552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:41.923930 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 438/852] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jun 15 19:25:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:25:41.925794 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 415/853] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jun 15 19:25:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:41.935052 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059bd59-1708-4808-a0c7-a93435037470 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:41.935177 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7b5b50-4010-499b-a502-7a2ffcd2edab None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:41.979943 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:42.021218 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:42.041388 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059bd59-1708-4808-a0c7-a93435037470 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:42.063529 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7b5b50-4010-499b-a502-7a2ffcd2edab None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:42.083757 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9bc1d-c0df-451a-9710-dfdb2f3f4e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:42.084457 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9bc1d-c0df-451a-9710-dfdb2f3f4e66 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:42.084740 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9bc1d-c0df-451a-9710-dfdb2f3f4e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:42.085014 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9bc1d-c0df-451a-9710-dfdb2f3f4e66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:42.085985 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da9bc1d-c0df-451a-9710-dfdb2f3f4e66 None admin] RBAC: Adding query filter params (role.id=4f736313c26d4f8db48bbb2f346e50e4, scope.project.id=7fb8cc0eb560465e972ffd45867ee73b, user.id=176d5775a70f4524a9b7fe1ce9d426ed) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:42.086599 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da9bc1d-c0df-451a-9710-dfdb2f3f4e66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:42.088109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da9bc1d-c0df-451a-9710-dfdb2f3f4e66 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:42.097575 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 416/854] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:25:41 2022] GET /identity/v3/role_assignments?user.id=176d5775a70f4524a9b7fe1ce9d426ed&scope.project.id=7fb8cc0eb560465e972ffd45867ee73b&role.id=4f736313c26d4f8db48bbb2f346e50e4 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:42.120618 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265f7a3-1af6-49f5-9043-e032cda784a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:42.121762 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265f7a3-1af6-49f5-9043-e032cda784a0 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:42.122321 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265f7a3-1af6-49f5-9043-e032cda784a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:42.122820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265f7a3-1af6-49f5-9043-e032cda784a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:42.123822 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a265f7a3-1af6-49f5-9043-e032cda784a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:42.124364 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a265f7a3-1af6-49f5-9043-e032cda784a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:42.126015 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a265f7a3-1af6-49f5-9043-e032cda784a0 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:42.136217 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 439/855] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jun 15 19:25:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:42.142187 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9bc1d-c0df-451a-9710-dfdb2f3f4e66 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:42.258536 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9bc1d-c0df-451a-9710-dfdb2f3f4e66 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:42.294723 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281147fe-2f6d-4b9d-b80f-bb2fc19ebeec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:42.295802 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281147fe-2f6d-4b9d-b80f-bb2fc19ebeec None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:42.296231 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281147fe-2f6d-4b9d-b80f-bb2fc19ebeec None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:42.296651 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281147fe-2f6d-4b9d-b80f-bb2fc19ebeec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:42.308482 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281147fe-2f6d-4b9d-b80f-bb2fc19ebeec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:42.309946 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281147fe-2f6d-4b9d-b80f-bb2fc19ebeec None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:42.327426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:42.331109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-281147fe-2f6d-4b9d-b80f-bb2fc19ebeec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:42.331109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:42.331109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:42.331109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:42.331109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:42.331109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:42.331109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:42.331109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:42.331109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:42.331109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:42.331109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:42.331109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:42.331109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:42.331109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:42.331109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:42.332028 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:42.332028 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:42.332028 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:42.332028 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:42.332028 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:42.332028 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:42.332028 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:42.332028 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:42.332028 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:42.332028 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:42.332028 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:42.332028 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:42.332028 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:42.332028 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:42.332028 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:42.332028 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:42.332028 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:42.332944 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:42.332944 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:42.332944 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:42.332944 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:42.332944 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 417/856] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:42.341154 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265f7a3-1af6-49f5-9043-e032cda784a0 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:42.396645 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb51a4-2f68-4b97-85e1-c5fa634731e0 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:42.396950 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb51a4-2f68-4b97-85e1-c5fa634731e0 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:42.397157 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb51a4-2f68-4b97-85e1-c5fa634731e0 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:42.398810 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 418/857] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 19:25:42.404184 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a0e0d-a5e2-4aa2-80ba-1d5509d7eee1 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:42.404469 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a0e0d-a5e2-4aa2-80ba-1d5509d7eee1 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:42.404747 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a0e0d-a5e2-4aa2-80ba-1d5509d7eee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:42.421753 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-7c5a0e0d-a5e2-4aa2-80ba-1d5509d7eee1 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:42.423763 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7c5a0e0d-a5e2-4aa2-80ba-1d5509d7eee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:42.450671 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:42.454113 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265f7a3-1af6-49f5-9043-e032cda784a0 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:42.497549 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef84478b-8502-4dec-bb84-15e1ccbcb7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:42.498339 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84478b-8502-4dec-bb84-15e1ccbcb7cb None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:42.498689 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84478b-8502-4dec-bb84-15e1ccbcb7cb None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:42.499032 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84478b-8502-4dec-bb84-15e1ccbcb7cb None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:42.499829 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef84478b-8502-4dec-bb84-15e1ccbcb7cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:42.500205 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef84478b-8502-4dec-bb84-15e1ccbcb7cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:42.506109 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef84478b-8502-4dec-bb84-15e1ccbcb7cb None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:42.506359 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 419/858] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:42 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 19:25:42.515282 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 440/859] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:42.520427 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a0e0d-a5e2-4aa2-80ba-1d5509d7eee1 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:42.526659 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef84478b-8502-4dec-bb84-15e1ccbcb7cb None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:42.638781 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef84478b-8502-4dec-bb84-15e1ccbcb7cb None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:42.654329 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a0e0d-a5e2-4aa2-80ba-1d5509d7eee1 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:42.677377 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f55a44-4efc-4edf-a4f8-431209a46cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:42.678727 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f55a44-4efc-4edf-a4f8-431209a46cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:42.679194 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f55a44-4efc-4edf-a4f8-431209a46cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:42.679670 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f55a44-4efc-4edf-a4f8-431209a46cd5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:42.680521 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f55a44-4efc-4edf-a4f8-431209a46cd5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:42.682650 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f55a44-4efc-4edf-a4f8-431209a46cd5 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:42.688822 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:42.697210 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-b3f55a44-4efc-4edf-a4f8-431209a46cd5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 19:25:42.697210 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:42.697210 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 19:25:42.697210 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 19:25:42.697210 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 19:25:42.697210 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 19:25:42.697210 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 19:25:42.697210 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:25:42.697210 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 19:25:42.697210 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:42.697210 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 19:25:42.697210 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:42.697210 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:42.697210 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:42.697210 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:42.697210 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:42.698205 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:42.698205 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:42.698205 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:42.698205 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:42.698205 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:42.698205 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:42.698205 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:42.698205 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 19:25:42.698205 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 19:25:42.698205 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 19:25:42.698205 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 19:25:42.698205 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:42.698205 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:42.698205 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:42.698205 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:42.698205 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:42.699056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:42.699056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:42.699056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:42.699056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:42.699056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:42.699056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:42.699056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:42.699056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:42.699056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:42.699056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:42.699056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:42.699056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 19:25:42.699056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:25:42.699056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 19:25:42.699056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:42.705304 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 441/860] 10.222.0.15 () {56 vars in 1132 bytes} [Wed Jun 15 19:25:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:42.705380 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5ca9c-19a5-4972-bd41-3df8468a003a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:42.705647 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5ca9c-19a5-4972-bd41-3df8468a003a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:42.705807 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5ca9c-19a5-4972-bd41-3df8468a003a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:42.706073 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5ca9c-19a5-4972-bd41-3df8468a003a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:42.707479 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f55a44-4efc-4edf-a4f8-431209a46cd5 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:42.717196 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e5ca9c-19a5-4972-bd41-3df8468a003a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:42.719232 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e5ca9c-19a5-4972-bd41-3df8468a003a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:42.737003 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:42.742395 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-17e5ca9c-19a5-4972-bd41-3df8468a003a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 19:25:42.742395 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:42.742395 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:42.742395 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:42.742395 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:42.742395 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:42.742395 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:42.742395 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:42.742395 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:42.742395 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:42.742395 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:42.742395 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:42.742395 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 19:25:42.742395 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 19:25:42.742395 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 19:25:42.742395 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 19:25:42.743351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:42.743351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:42.743351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:25:42.743351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:42.743351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:25:42.743351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:42.743351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:42.743351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:42.743351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:42.743351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:42.743351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:42.743351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:42.743351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:42.743351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:42.743351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:42.743351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:42.743351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:42.743351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:42.744430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:42.744430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:42.744430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 19:25:42.744430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 19:25:42.744430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 19:25:42.744430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 19:25:42.744430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 19:25:42.744430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 19:25:42.744430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 19:25:42.744430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:42.746163 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 420/861] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Jun 15 19:25:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:42.749217 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5ca9c-19a5-4972-bd41-3df8468a003a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:42.831729 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f55a44-4efc-4edf-a4f8-431209a46cd5 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:42.868551 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a60c0b-3de2-4711-bdb7-9562fb8e6185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:42.869174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5ca9c-19a5-4972-bd41-3df8468a003a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:42.869866 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a60c0b-3de2-4711-bdb7-9562fb8e6185 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:42.870380 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a60c0b-3de2-4711-bdb7-9562fb8e6185 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:42.870865 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a60c0b-3de2-4711-bdb7-9562fb8e6185 None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:42.871781 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a60c0b-3de2-4711-bdb7-9562fb8e6185 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:42.872297 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a60c0b-3de2-4711-bdb7-9562fb8e6185 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:42.873890 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a60c0b-3de2-4711-bdb7-9562fb8e6185 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:42.884638 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 442/862] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Jun 15 19:25:42 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:42.885892 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a60c0b-3de2-4711-bdb7-9562fb8e6185 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:42.908594 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78601be6-5bfd-4807-91a2-49c9b9ad4205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:42.909196 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78601be6-5bfd-4807-91a2-49c9b9ad4205 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:42.909898 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78601be6-5bfd-4807-91a2-49c9b9ad4205 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:42.909898 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78601be6-5bfd-4807-91a2-49c9b9ad4205 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:42.910753 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78601be6-5bfd-4807-91a2-49c9b9ad4205 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:42.911062 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78601be6-5bfd-4807-91a2-49c9b9ad4205 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:42.912482 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78601be6-5bfd-4807-91a2-49c9b9ad4205 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:42.932232 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 421/863] 10.222.0.15 () {56 vars in 1149 bytes} [Wed Jun 15 19:25:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:25:42.937848 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78601be6-5bfd-4807-91a2-49c9b9ad4205 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:42.981311 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:42.990701 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:42.995497 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a60c0b-3de2-4711-bdb7-9562fb8e6185 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:43.031329 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee2130-fefd-4e7a-b78f-2ba2ecf2cb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:43.032138 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee2130-fefd-4e7a-b78f-2ba2ecf2cb56 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:43.032524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee2130-fefd-4e7a-b78f-2ba2ecf2cb56 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:43.032903 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee2130-fefd-4e7a-b78f-2ba2ecf2cb56 None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:43.033740 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ee2130-fefd-4e7a-b78f-2ba2ecf2cb56 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:43.034181 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ee2130-fefd-4e7a-b78f-2ba2ecf2cb56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:43.035642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ee2130-fefd-4e7a-b78f-2ba2ecf2cb56 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:43.046379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 443/864] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Jun 15 19:25:42 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:43.052478 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78601be6-5bfd-4807-91a2-49c9b9ad4205 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:43.097537 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf789e3-e12d-429a-9b2b-6c766e99ff0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:43.099003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf789e3-e12d-429a-9b2b-6c766e99ff0f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:43.099442 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf789e3-e12d-429a-9b2b-6c766e99ff0f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:43.099861 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf789e3-e12d-429a-9b2b-6c766e99ff0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:43.141541 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b1cc0-9519-49c7-a2af-377645d7529a None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:43.142075 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b1cc0-9519-49c7-a2af-377645d7529a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:43.142075 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b1cc0-9519-49c7-a2af-377645d7529a None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:43.147880 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 444/865] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 19:25:43.149049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52984905-b1ed-4bd1-918f-0581b61336bb None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:43.150411 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52984905-b1ed-4bd1-918f-0581b61336bb None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:43.150843 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52984905-b1ed-4bd1-918f-0581b61336bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:43.174971 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-52984905-b1ed-4bd1-918f-0581b61336bb None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:43.176457 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-52984905-b1ed-4bd1-918f-0581b61336bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:43.181097 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf789e3-e12d-429a-9b2b-6c766e99ff0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:43.182768 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf789e3-e12d-429a-9b2b-6c766e99ff0f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:43.232299 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:43.275246 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 445/866] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:43 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 19:25:43.297508 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52984905-b1ed-4bd1-918f-0581b61336bb None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:43.299918 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 422/867] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:25:42 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 19:25:43.304737 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf789e3-e12d-429a-9b2b-6c766e99ff0f None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:43.409329 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:43.420958 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf789e3-e12d-429a-9b2b-6c766e99ff0f None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:43.435364 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52984905-b1ed-4bd1-918f-0581b61336bb None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:43.467369 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2cd858-be80-4db6-92ab-4611a64066f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:43.468169 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2cd858-be80-4db6-92ab-4611a64066f2 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:43.468452 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2cd858-be80-4db6-92ab-4611a64066f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:43.468721 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2cd858-be80-4db6-92ab-4611a64066f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:43.483397 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4571563-63e3-4c6f-a2ae-d8723f257149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:43.484748 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4571563-63e3-4c6f-a2ae-d8723f257149 None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:43.484863 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2cd858-be80-4db6-92ab-4611a64066f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:43.485426 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4571563-63e3-4c6f-a2ae-d8723f257149 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:43.485954 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4571563-63e3-4c6f-a2ae-d8723f257149 None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:43.486411 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2cd858-be80-4db6-92ab-4611a64066f2 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:43.487147 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4571563-63e3-4c6f-a2ae-d8723f257149 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:43.489366 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4571563-63e3-4c6f-a2ae-d8723f257149 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:43.494981 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:43.506911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-5e2cd858-be80-4db6-92ab-4611a64066f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:43.506911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:43.506911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:43.506911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:43.506911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:43.506911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:43.506911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:43.506911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:43.506911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:43.506911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:43.506911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:43.506911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:43.506911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:43.506911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:43.506911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:43.508392 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:43.508392 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:43.508392 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:43.508392 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:43.508392 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:43.508392 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:43.508392 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:43.508392 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:43.508392 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:43.508392 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:43.508392 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:43.508392 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:43.508392 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:43.508392 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:43.508392 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:43.508392 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:43.508392 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:43.509692 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:43.509692 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:43.509692 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:43.509692 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:43.510601 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 446/869] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Jun 15 19:25:43 2022] POST /identity/v3/services => generated 251 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:25:43.513127 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 423/869] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:43.514359 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4571563-63e3-4c6f-a2ae-d8723f257149 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:43.637848 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4571563-63e3-4c6f-a2ae-d8723f257149 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:43.676654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e6cf7-b428-4413-88c2-3322a8d70c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:43.677325 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e6cf7-b428-4413-88c2-3322a8d70c46 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:43.677609 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e6cf7-b428-4413-88c2-3322a8d70c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:43.677888 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e6cf7-b428-4413-88c2-3322a8d70c46 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:43.678656 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e6cf7-b428-4413-88c2-3322a8d70c46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:43.678966 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e6cf7-b428-4413-88c2-3322a8d70c46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:43.680328 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e6cf7-b428-4413-88c2-3322a8d70c46 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:43.687766 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:43.693354 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 447/870] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:25:43.699260 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2cd858-be80-4db6-92ab-4611a64066f2 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:43.824804 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2cd858-be80-4db6-92ab-4611a64066f2 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:43.863327 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30d59b6-9364-4d66-9736-8440504027bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:43.864132 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d59b6-9364-4d66-9736-8440504027bb None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:43.864414 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d59b6-9364-4d66-9736-8440504027bb None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:43.864690 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d59b6-9364-4d66-9736-8440504027bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:43.872363 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30d59b6-9364-4d66-9736-8440504027bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:43.873816 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30d59b6-9364-4d66-9736-8440504027bb None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:43.888210 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-a30d59b6-9364-4d66-9736-8440504027bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:43.888210 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:43.888210 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:43.888210 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:43.888210 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:43.888210 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:43.888210 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:43.888210 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:43.888210 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:43.888210 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:43.888210 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:43.888210 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:43.888210 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 19:25:43.888210 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 19:25:43.888210 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 19:25:43.889154 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 19:25:43.889154 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:43.889154 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:43.889154 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:43.889154 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:43.889154 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:43.889154 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:43.889154 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:43.889154 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:43.889154 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:43.889154 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:43.889154 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:43.889154 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:43.889154 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:43.889154 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:43.889154 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:43.889154 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:43.890064 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 19:25:43.890064 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 19:25:43.890064 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:25:43.890064 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:25:43.890064 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:25:43.890064 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:25:43.890064 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:43.890064 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:43.890064 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 424/871] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:25:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:43.895524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e6cf7-b428-4413-88c2-3322a8d70c46 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:44.005316 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e6cf7-b428-4413-88c2-3322a8d70c46 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:44.043506 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd3a72a-77a4-460e-af2e-5610ccaadcdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:44.044177 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3a72a-77a4-460e-af2e-5610ccaadcdd None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:44.044465 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3a72a-77a4-460e-af2e-5610ccaadcdd None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:44.044741 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3a72a-77a4-460e-af2e-5610ccaadcdd None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:44.045544 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd3a72a-77a4-460e-af2e-5610ccaadcdd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:44.045853 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd3a72a-77a4-460e-af2e-5610ccaadcdd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:44.047320 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd3a72a-77a4-460e-af2e-5610ccaadcdd None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:44.063761 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 448/872] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jun 15 19:25:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:44.069832 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30d59b6-9364-4d66-9736-8440504027bb None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:44.135330 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:44.178403 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30d59b6-9364-4d66-9736-8440504027bb None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:44.215396 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22ef62d-0a5b-4e8e-bda7-dd543b7f4ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:44.216057 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ef62d-0a5b-4e8e-bda7-dd543b7f4ad4 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:44.216382 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ef62d-0a5b-4e8e-bda7-dd543b7f4ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:44.216669 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ef62d-0a5b-4e8e-bda7-dd543b7f4ad4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:44.217579 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22ef62d-0a5b-4e8e-bda7-dd543b7f4ad4 None admin] RBAC: Adding query filter params (role.id=a8df9714563b435e9fce55a04e74ae5e, scope.project.id=7fb8cc0eb560465e972ffd45867ee73b, user.id=67aa048d409c46e9a729aec9d1b2286e) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:44.217892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22ef62d-0a5b-4e8e-bda7-dd543b7f4ad4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:44.219392 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22ef62d-0a5b-4e8e-bda7-dd543b7f4ad4 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:44.225727 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 425/873] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:25:44 2022] GET /identity/v3/role_assignments?user.id=67aa048d409c46e9a729aec9d1b2286e&scope.project.id=7fb8cc0eb560465e972ffd45867ee73b&role.id=a8df9714563b435e9fce55a04e74ae5e => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:44.353023 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc653b7d-25ab-40f9-b967-feba280ff1d0 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:44.353558 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc653b7d-25ab-40f9-b967-feba280ff1d0 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:44.353831 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc653b7d-25ab-40f9-b967-feba280ff1d0 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:44.355664 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 449/874] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:44.361408 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e43c7b-597e-4649-badd-8cf10859744b None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:44.361836 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e43c7b-597e-4649-badd-8cf10859744b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:44.362266 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e43c7b-597e-4649-badd-8cf10859744b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:44.380054 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-48e43c7b-597e-4649-badd-8cf10859744b None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:44.381268 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-48e43c7b-597e-4649-badd-8cf10859744b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:44.457657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 426/875] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:44 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:25:44.479792 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc653b7d-25ab-40f9-b967-feba280ff1d0 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:44.543608 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:44.586600 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc653b7d-25ab-40f9-b967-feba280ff1d0 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:44.625118 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66b98e-1e05-4ffc-9e20-2af87d940114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:44.626275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66b98e-1e05-4ffc-9e20-2af87d940114 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:44.626773 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66b98e-1e05-4ffc-9e20-2af87d940114 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:44.627273 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66b98e-1e05-4ffc-9e20-2af87d940114 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:44.628213 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66b98e-1e05-4ffc-9e20-2af87d940114 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:44.629893 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66b98e-1e05-4ffc-9e20-2af87d940114 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:44.643043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-7c66b98e-1e05-4ffc-9e20-2af87d940114 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 19:25:44.643043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:44.643043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 19:25:44.643043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 19:25:44.643043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 19:25:44.643043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 19:25:44.643043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 19:25:44.643043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:25:44.643043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 19:25:44.643043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:44.643043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 19:25:44.643043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:44.643043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:44.643043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:44.643043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:44.643043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:44.643043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:44.644074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:44.644074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:44.644074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:44.644074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:44.644074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:44.644074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:44.644074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 19:25:44.644074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 19:25:44.644074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 19:25:44.644074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 19:25:44.644074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:44.644074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:44.644074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:44.644074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:44.644074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:44.644074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:44.644956 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:44.644956 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:44.644956 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:44.644956 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:44.644956 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:44.644956 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:44.644956 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:44.644956 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:44.644956 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:44.644956 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:44.644956 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 19:25:44.644956 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:25:44.644956 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 19:25:44.644956 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:44.647892 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 450/876] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Jun 15 19:25:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:44.653439 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e43c7b-597e-4649-badd-8cf10859744b None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:44.706933 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:44.760383 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e43c7b-597e-4649-badd-8cf10859744b None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:44.796383 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94947eb-6073-42c7-b677-f854c71b200c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:44.797055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94947eb-6073-42c7-b677-f854c71b200c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:44.797340 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94947eb-6073-42c7-b677-f854c71b200c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:44.797613 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94947eb-6073-42c7-b677-f854c71b200c None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:44.798353 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94947eb-6073-42c7-b677-f854c71b200c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:44.798659 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94947eb-6073-42c7-b677-f854c71b200c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:44.800032 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94947eb-6073-42c7-b677-f854c71b200c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:44.805309 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 427/877] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Jun 15 19:25:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:44.811274 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66b98e-1e05-4ffc-9e20-2af87d940114 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:44.914410 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66b98e-1e05-4ffc-9e20-2af87d940114 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:44.946463 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:44.954119 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b30474-28ff-4add-a870-6b5f2685b4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:44.954788 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b30474-28ff-4add-a870-6b5f2685b4fd None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:44.955076 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b30474-28ff-4add-a870-6b5f2685b4fd None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:44.955351 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b30474-28ff-4add-a870-6b5f2685b4fd None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:44.956062 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b30474-28ff-4add-a870-6b5f2685b4fd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:44.956370 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b30474-28ff-4add-a870-6b5f2685b4fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:44.957731 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b30474-28ff-4add-a870-6b5f2685b4fd None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:44.963300 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 451/878] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Jun 15 19:25:44 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:45.273775 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb394c-ef46-4aa2-9e36-0c6590f4e1e1 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:45.274392 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb394c-ef46-4aa2-9e36-0c6590f4e1e1 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:45.274392 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb394c-ef46-4aa2-9e36-0c6590f4e1e1 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:45.276190 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 428/879] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:45.281374 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1318c7-105e-41b8-b159-5a6d97d4fcd7 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:45.281664 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1318c7-105e-41b8-b159-5a6d97d4fcd7 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:45.281943 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1318c7-105e-41b8-b159-5a6d97d4fcd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:45.299387 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-ee1318c7-105e-41b8-b159-5a6d97d4fcd7 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:45.300577 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-ee1318c7-105e-41b8-b159-5a6d97d4fcd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:45.369919 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 452/880] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:45 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:25:45.392822 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb394c-ef46-4aa2-9e36-0c6590f4e1e1 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:45.500283 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb394c-ef46-4aa2-9e36-0c6590f4e1e1 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:45.537560 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a12bb03-532b-442f-a417-c2593236baf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:45.538338 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12bb03-532b-442f-a417-c2593236baf9 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:45.538743 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12bb03-532b-442f-a417-c2593236baf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:45.539116 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12bb03-532b-442f-a417-c2593236baf9 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:45.539974 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a12bb03-532b-442f-a417-c2593236baf9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:45.541538 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a12bb03-532b-442f-a417-c2593236baf9 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:45.546496 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:45.562258 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 429/881] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Jun 15 19:25:45 2022] POST /identity/v3/services => generated 269 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:25:45.823794 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef91ad-9e75-4485-ad47-62b88813b573 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:45.823986 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef91ad-9e75-4485-ad47-62b88813b573 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:45.824254 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef91ad-9e75-4485-ad47-62b88813b573 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:45.825901 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 453/882] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:45.831085 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d9e68-dc09-44fa-8d9c-7da98ea76123 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:45.831377 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d9e68-dc09-44fa-8d9c-7da98ea76123 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:45.831658 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d9e68-dc09-44fa-8d9c-7da98ea76123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:45.851850 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-213d9e68-dc09-44fa-8d9c-7da98ea76123 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:45.853134 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-213d9e68-dc09-44fa-8d9c-7da98ea76123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:45.928328 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 430/883] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:45 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:25:45.951523 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef91ad-9e75-4485-ad47-62b88813b573 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:46.054915 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef91ad-9e75-4485-ad47-62b88813b573 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:46.078122 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:46.096666 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d09ddd-f0ba-4864-8731-19eb936af648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:46.097466 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d09ddd-f0ba-4864-8731-19eb936af648 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:46.097752 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d09ddd-f0ba-4864-8731-19eb936af648 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:46.098030 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d09ddd-f0ba-4864-8731-19eb936af648 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:46.098719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d09ddd-f0ba-4864-8731-19eb936af648 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:46.100098 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d09ddd-f0ba-4864-8731-19eb936af648 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:46.112786 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-b4d09ddd-f0ba-4864-8731-19eb936af648 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 19:25:46.112786 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:46.112786 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 19:25:46.112786 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 19:25:46.112786 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 19:25:46.112786 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 19:25:46.112786 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 19:25:46.112786 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:25:46.112786 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 19:25:46.112786 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:46.112786 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 19:25:46.112786 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:46.112786 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:46.112786 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:46.112786 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:46.112786 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:46.112786 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:46.113783 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:46.113783 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:46.113783 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:46.113783 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:46.113783 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:46.113783 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:46.113783 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 19:25:46.113783 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 19:25:46.113783 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 19:25:46.113783 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 19:25:46.113783 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:46.113783 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:46.113783 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:46.113783 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:46.113783 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:46.113783 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:46.114794 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:46.114794 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:46.114794 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:46.114794 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:46.114794 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:46.114794 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:46.114794 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:46.114794 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:46.114794 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:46.114794 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:46.114794 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 19:25:46.114794 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:25:46.114794 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 19:25:46.114794 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:46.114794 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 454/884] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jun 15 19:25:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:46.120499 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d9e68-dc09-44fa-8d9c-7da98ea76123 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:46.222783 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d9e68-dc09-44fa-8d9c-7da98ea76123 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:46.224324 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:46.261283 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45e8c0-2106-417b-a32b-bb190b167081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:46.262294 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45e8c0-2106-417b-a32b-bb190b167081 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:46.262739 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45e8c0-2106-417b-a32b-bb190b167081 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:46.263160 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45e8c0-2106-417b-a32b-bb190b167081 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:46.264009 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a45e8c0-2106-417b-a32b-bb190b167081 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:46.264457 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a45e8c0-2106-417b-a32b-bb190b167081 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:46.265981 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a45e8c0-2106-417b-a32b-bb190b167081 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:46.271150 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 431/885] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jun 15 19:25:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:46.277349 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d09ddd-f0ba-4864-8731-19eb936af648 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:46.394577 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d09ddd-f0ba-4864-8731-19eb936af648 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:46.428913 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159a6e9-6043-426a-b2ef-9c0cd8f15f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:46.429579 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159a6e9-6043-426a-b2ef-9c0cd8f15f9e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:46.429863 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159a6e9-6043-426a-b2ef-9c0cd8f15f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:46.430165 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159a6e9-6043-426a-b2ef-9c0cd8f15f9e None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:46.431406 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b159a6e9-6043-426a-b2ef-9c0cd8f15f9e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:46.431719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b159a6e9-6043-426a-b2ef-9c0cd8f15f9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:46.433088 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b159a6e9-6043-426a-b2ef-9c0cd8f15f9e None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:46.438530 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 455/886] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jun 15 19:25:46 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:46.447478 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45e8c0-2106-417b-a32b-bb190b167081 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:46.559807 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45e8c0-2106-417b-a32b-bb190b167081 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:46.600514 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c51a48-6ce9-4ed7-8e8d-5dc001b31320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:46.601509 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c51a48-6ce9-4ed7-8e8d-5dc001b31320 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:46.601994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c51a48-6ce9-4ed7-8e8d-5dc001b31320 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:46.602564 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c51a48-6ce9-4ed7-8e8d-5dc001b31320 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:46.603584 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c51a48-6ce9-4ed7-8e8d-5dc001b31320 None admin] RBAC: Adding query filter params (interface=public, service_id=3ff28bdde2644f9fab4bc217c20c6ba6, region_id=RegionOne) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:46.604114 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c51a48-6ce9-4ed7-8e8d-5dc001b31320 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:46.605674 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c51a48-6ce9-4ed7-8e8d-5dc001b31320 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:46.611606 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 432/887] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Jun 15 19:25:46 2022] GET /identity/v3/endpoints?service_id=3ff28bdde2644f9fab4bc217c20c6ba6&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:46.617238 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159a6e9-6043-426a-b2ef-9c0cd8f15f9e None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:46.629682 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a970b44-bfc5-4f49-8a27-6349437ca5f3 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:46.630354 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a970b44-bfc5-4f49-8a27-6349437ca5f3 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:46.630837 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a970b44-bfc5-4f49-8a27-6349437ca5f3 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:46.632626 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 433/888] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 19:25:46.638831 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d3f95-38f2-40f8-8f3b-2fc264e5bba4 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:46.639330 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d3f95-38f2-40f8-8f3b-2fc264e5bba4 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:46.640049 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d3f95-38f2-40f8-8f3b-2fc264e5bba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:46.661989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-b58d3f95-38f2-40f8-8f3b-2fc264e5bba4 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:46.663527 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b58d3f95-38f2-40f8-8f3b-2fc264e5bba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:46.672510 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:46.695724 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:46.733539 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159a6e9-6043-426a-b2ef-9c0cd8f15f9e None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:46.751025 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 434/889] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 19:25:46.775212 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d3f95-38f2-40f8-8f3b-2fc264e5bba4 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:46.775633 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f47e3-1333-471c-ba3c-4b8bdbce6154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:46.776436 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f47e3-1333-471c-ba3c-4b8bdbce6154 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:46.776820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f47e3-1333-471c-ba3c-4b8bdbce6154 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:46.777191 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f47e3-1333-471c-ba3c-4b8bdbce6154 None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:46.777981 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7f47e3-1333-471c-ba3c-4b8bdbce6154 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:46.780064 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7f47e3-1333-471c-ba3c-4b8bdbce6154 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:46.790452 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 456/890] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Jun 15 19:25:46 2022] GET /identity/v3/services => generated 1268 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:25:46.893062 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d3f95-38f2-40f8-8f3b-2fc264e5bba4 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:46.930446 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cea7ebf-840f-46db-a488-2f7f70707ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:46.931331 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea7ebf-840f-46db-a488-2f7f70707ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:46.931610 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea7ebf-840f-46db-a488-2f7f70707ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:46.931880 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea7ebf-840f-46db-a488-2f7f70707ee1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:46.943805 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cea7ebf-840f-46db-a488-2f7f70707ee1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:25:46.952943 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-4cea7ebf-840f-46db-a488-2f7f70707ee1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 19:25:46.952943 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:46.952943 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:46.952943 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:46.952943 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:46.952943 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:46.952943 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:46.952943 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:46.952943 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:46.952943 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:46.952943 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:46.952943 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:46.952943 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:25:46.952943 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:25:46.952943 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:25:46.952943 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:25:46.953922 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:25:46.953922 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:25:46.953922 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:25:46.953922 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:25:46.953922 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:46.953922 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:46.953922 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:46.953922 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:46.953922 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:46.953922 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:46.953922 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:46.953922 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:46.953922 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:46.953922 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:46.953922 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:46.953922 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:46.954795 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:46.954795 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:46.954795 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:46.954795 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:46.954795 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:25:46.954795 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:25:46.954795 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:25:46.954795 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:25:46.954795 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:25:46.954795 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:25:46.954795 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 19:25:46.954795 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:46.954795 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 435/891] 10.222.0.15 () {56 vars in 1120 bytes} [Wed Jun 15 19:25:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:46.960314 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f47e3-1333-471c-ba3c-4b8bdbce6154 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:47.083119 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f47e3-1333-471c-ba3c-4b8bdbce6154 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:47.093142 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:47.113802 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b7538-6ca7-44da-8f00-5e32abd25199 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:47.114135 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b7538-6ca7-44da-8f00-5e32abd25199 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:47.114396 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b7538-6ca7-44da-8f00-5e32abd25199 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:47.116040 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 436/892] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:47.121575 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcaae28-3cef-43db-b530-375f3173d64c None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:47.121872 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcaae28-3cef-43db-b530-375f3173d64c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:47.122176 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcaae28-3cef-43db-b530-375f3173d64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:47.123814 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0781d443-4b41-4e16-baf7-4834d1e5dce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:47.124702 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781d443-4b41-4e16-baf7-4834d1e5dce1 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:47.125197 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781d443-4b41-4e16-baf7-4834d1e5dce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:47.125687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781d443-4b41-4e16-baf7-4834d1e5dce1 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:47.126681 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0781d443-4b41-4e16-baf7-4834d1e5dce1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:47.127201 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0781d443-4b41-4e16-baf7-4834d1e5dce1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:47.128767 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0781d443-4b41-4e16-baf7-4834d1e5dce1 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:47.142178 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 457/893] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Jun 15 19:25:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:47.144075 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-dbcaae28-3cef-43db-b530-375f3173d64c None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:47.145200 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-dbcaae28-3cef-43db-b530-375f3173d64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:47.147031 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0781d443-4b41-4e16-baf7-4834d1e5dce1 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:47.161098 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:47.242427 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 437/894] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:47 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 19:25:47.266645 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcaae28-3cef-43db-b530-375f3173d64c None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:47.267768 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0781d443-4b41-4e16-baf7-4834d1e5dce1 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:47.303101 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e99159-24e1-43c3-8acb-fb1735cd6323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:47.304376 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e99159-24e1-43c3-8acb-fb1735cd6323 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:47.304886 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e99159-24e1-43c3-8acb-fb1735cd6323 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:47.306014 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e99159-24e1-43c3-8acb-fb1735cd6323 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:47.318112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e99159-24e1-43c3-8acb-fb1735cd6323 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:47.319901 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e99159-24e1-43c3-8acb-fb1735cd6323 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:47.339199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-29e99159-24e1-43c3-8acb-fb1735cd6323 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:47.339199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:47.339199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:47.339199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:47.339199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:47.339199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:47.339199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:47.339199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:47.339199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:47.339199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:47.339199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:47.339199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:47.339199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:47.339199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:47.339199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:47.340164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:47.340164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:47.340164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:47.340164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:47.340164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:47.340164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:47.340164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:47.340164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:47.340164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:47.340164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:47.340164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:47.340164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:47.340164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:47.340164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:47.340164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:47.340164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:47.340164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:47.341053 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:47.341053 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:47.341053 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:47.341053 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:47.347989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 458/895] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:47.349256 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e99159-24e1-43c3-8acb-fb1735cd6323 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:47.376668 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:47.408117 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcaae28-3cef-43db-b530-375f3173d64c None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:47.443620 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305c3201-d58d-4f17-add3-4c30c8f97915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:47.444559 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c3201-d58d-4f17-add3-4c30c8f97915 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:47.444985 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c3201-d58d-4f17-add3-4c30c8f97915 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:47.445404 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c3201-d58d-4f17-add3-4c30c8f97915 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:47.446909 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305c3201-d58d-4f17-add3-4c30c8f97915 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:47.448474 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305c3201-d58d-4f17-add3-4c30c8f97915 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:47.463010 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e99159-24e1-43c3-8acb-fb1735cd6323 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:47.467339 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 438/896] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Jun 15 19:25:47 2022] POST /identity/v3/services => generated 241 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:25:47.511290 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e06a0-ce56-4cee-9a6b-699bd3b21734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:47.512260 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e06a0-ce56-4cee-9a6b-699bd3b21734 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:47.512729 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e06a0-ce56-4cee-9a6b-699bd3b21734 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:47.513152 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e06a0-ce56-4cee-9a6b-699bd3b21734 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:47.514028 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e06a0-ce56-4cee-9a6b-699bd3b21734 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:47.514508 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e06a0-ce56-4cee-9a6b-699bd3b21734 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:47.516126 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e06a0-ce56-4cee-9a6b-699bd3b21734 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:47.531842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 459/897] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:47.538185 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305c3201-d58d-4f17-add3-4c30c8f97915 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:47.558540 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:47.658494 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305c3201-d58d-4f17-add3-4c30c8f97915 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:47.694066 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f6d9e6-645d-4a0a-9bfe-72ad6dc96d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:47.694945 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6d9e6-645d-4a0a-9bfe-72ad6dc96d04 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:47.695226 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6d9e6-645d-4a0a-9bfe-72ad6dc96d04 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:47.695498 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6d9e6-645d-4a0a-9bfe-72ad6dc96d04 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:47.704276 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f6d9e6-645d-4a0a-9bfe-72ad6dc96d04 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:47.705716 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f6d9e6-645d-4a0a-9bfe-72ad6dc96d04 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:47.723479 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-a5f6d9e6-645d-4a0a-9bfe-72ad6dc96d04 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 19:25:47.723479 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:47.723479 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:47.723479 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:47.723479 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:47.723479 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:47.723479 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:47.723479 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:47.723479 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:47.723479 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:47.723479 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:47.723479 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:47.723479 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 19:25:47.723479 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 19:25:47.723479 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 19:25:47.723479 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 19:25:47.724480 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:47.724480 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:47.724480 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:25:47.724480 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:47.724480 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:25:47.724480 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:47.724480 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:47.724480 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:47.724480 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:47.724480 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:47.724480 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:47.724480 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:47.724480 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:47.724480 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:47.724480 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:47.724480 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:47.724480 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:47.724480 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:47.725441 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:47.725441 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:47.725441 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 19:25:47.725441 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 19:25:47.725441 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 19:25:47.725441 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 19:25:47.725441 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 19:25:47.725441 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 19:25:47.725441 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 19:25:47.725441 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:47.725441 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 439/898] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Jun 15 19:25:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:47.730484 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e06a0-ce56-4cee-9a6b-699bd3b21734 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:47.818034 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:47.834762 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba116308-98df-48a4-bd38-eb039660687a None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:47.835089 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba116308-98df-48a4-bd38-eb039660687a None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:47.835322 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba116308-98df-48a4-bd38-eb039660687a None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:47.838003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 440/899] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:47.841880 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c630377-81c7-4d84-b1a1-1f63aec75b7c None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:47.842206 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c630377-81c7-4d84-b1a1-1f63aec75b7c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:47.842485 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c630377-81c7-4d84-b1a1-1f63aec75b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:47.844377 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e06a0-ce56-4cee-9a6b-699bd3b21734 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:47.859995 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-8c630377-81c7-4d84-b1a1-1f63aec75b7c None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:47.861179 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8c630377-81c7-4d84-b1a1-1f63aec75b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:47.890080 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d245b-14b0-49db-bdd2-e05c6493da68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:47.891318 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d245b-14b0-49db-bdd2-e05c6493da68 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:47.892032 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d245b-14b0-49db-bdd2-e05c6493da68 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:47.892501 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d245b-14b0-49db-bdd2-e05c6493da68 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:47.894139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d245b-14b0-49db-bdd2-e05c6493da68 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:47.894736 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d245b-14b0-49db-bdd2-e05c6493da68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:47.896347 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d245b-14b0-49db-bdd2-e05c6493da68 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:47.920835 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 460/900] 10.222.0.15 () {56 vars in 1149 bytes} [Wed Jun 15 19:25:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:47.925719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d245b-14b0-49db-bdd2-e05c6493da68 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:47.929641 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:47.942262 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 441/901] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 19:25:47.959338 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c630377-81c7-4d84-b1a1-1f63aec75b7c None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:48.064696 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d245b-14b0-49db-bdd2-e05c6493da68 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:48.076925 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c630377-81c7-4d84-b1a1-1f63aec75b7c None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:48.104026 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc67b0-b8df-4597-a525-64613afd345b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:48.104770 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc67b0-b8df-4597-a525-64613afd345b None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:48.105118 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc67b0-b8df-4597-a525-64613afd345b None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:48.105478 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc67b0-b8df-4597-a525-64613afd345b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:48.109874 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:48.129920 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47b1a92-f1cb-4804-ae23-71ff0b43642d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:48.130979 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b1a92-f1cb-4804-ae23-71ff0b43642d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:48.131977 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b1a92-f1cb-4804-ae23-71ff0b43642d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:48.132409 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b1a92-f1cb-4804-ae23-71ff0b43642d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:48.134296 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47b1a92-f1cb-4804-ae23-71ff0b43642d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:48.136485 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47b1a92-f1cb-4804-ae23-71ff0b43642d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:48.152892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-b47b1a92-f1cb-4804-ae23-71ff0b43642d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 19:25:48.152892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:48.152892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 19:25:48.152892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 19:25:48.152892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 19:25:48.152892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 19:25:48.152892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 19:25:48.152892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:25:48.152892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 19:25:48.152892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:48.152892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 19:25:48.152892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:48.152892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:48.152892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:48.152892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:48.152892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:48.154081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:48.154081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:48.154081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:48.154081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:48.154081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:48.154081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:48.154081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:48.154081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 19:25:48.154081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 19:25:48.154081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 19:25:48.154081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 19:25:48.154081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:48.154081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:48.154081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:48.154081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:48.154081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:48.155168 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:48.155168 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:48.155168 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:48.155168 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:48.155168 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:48.155168 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:48.155168 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:48.155168 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:48.155168 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:48.155168 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:48.155168 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:48.155168 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 19:25:48.155168 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:25:48.155168 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 19:25:48.155168 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:48.158325 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 442/902] 10.222.0.15 () {56 vars in 1132 bytes} [Wed Jun 15 19:25:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:25:48.164137 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47b1a92-f1cb-4804-ae23-71ff0b43642d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:48.189553 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc67b0-b8df-4597-a525-64613afd345b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:48.191097 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc67b0-b8df-4597-a525-64613afd345b None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:48.234793 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:48.274863 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47b1a92-f1cb-4804-ae23-71ff0b43642d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:48.310160 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ab090-fa46-4ee4-bae4-64b08ca5dbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:48.310973 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ab090-fa46-4ee4-bae4-64b08ca5dbaf None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:48.311372 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ab090-fa46-4ee4-bae4-64b08ca5dbaf None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:48.311742 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ab090-fa46-4ee4-bae4-64b08ca5dbaf None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:48.312971 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ab090-fa46-4ee4-bae4-64b08ca5dbaf None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:48.313377 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ab090-fa46-4ee4-bae4-64b08ca5dbaf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:48.315119 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ab090-fa46-4ee4-bae4-64b08ca5dbaf None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:48.326238 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 443/903] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Jun 15 19:25:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:25:48.328901 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ab090-fa46-4ee4-bae4-64b08ca5dbaf None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:48.334938 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 461/904] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:25:47 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 15 19:25:48.342148 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc67b0-b8df-4597-a525-64613afd345b None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:48.406535 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:48.461350 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc67b0-b8df-4597-a525-64613afd345b None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:48.463634 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ab090-fa46-4ee4-bae4-64b08ca5dbaf None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:48.497209 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8632edd-0215-48b6-8ce7-75a287a94cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:48.498117 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8632edd-0215-48b6-8ce7-75a287a94cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:48.498482 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8632edd-0215-48b6-8ce7-75a287a94cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:48.498829 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8632edd-0215-48b6-8ce7-75a287a94cb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:48.499715 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:48.510994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c098211a-81ee-41ed-8be7-f0b938c7004a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:48.512195 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098211a-81ee-41ed-8be7-f0b938c7004a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:48.512313 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8632edd-0215-48b6-8ce7-75a287a94cb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:48.512798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098211a-81ee-41ed-8be7-f0b938c7004a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:48.513254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098211a-81ee-41ed-8be7-f0b938c7004a None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:48.513931 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8632edd-0215-48b6-8ce7-75a287a94cb6 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:48.514354 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c098211a-81ee-41ed-8be7-f0b938c7004a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:48.514843 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c098211a-81ee-41ed-8be7-f0b938c7004a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:48.517016 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c098211a-81ee-41ed-8be7-f0b938c7004a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:48.528133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 444/905] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Jun 15 19:25:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:25:48.530229 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c098211a-81ee-41ed-8be7-f0b938c7004a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:48.540081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-b8632edd-0215-48b6-8ce7-75a287a94cb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:48.540081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:48.540081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:48.540081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:48.540081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:48.540081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:48.540081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:48.540081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:48.540081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:48.540081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:48.540081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:48.540081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:48.540081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:48.540081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:48.540081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:48.541055 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:48.541055 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:48.541055 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:48.541055 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:48.541055 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:48.541055 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:48.541055 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:48.541055 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:48.541055 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:48.541055 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:48.541055 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:48.541055 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:48.541055 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:48.541055 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:48.541055 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:48.541055 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:48.541055 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:48.542023 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:48.542023 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:48.542023 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:48.542023 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:48.546192 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 462/906] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:48.549596 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8632edd-0215-48b6-8ce7-75a287a94cb6 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:48.647935 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c098211a-81ee-41ed-8be7-f0b938c7004a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:48.668953 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:48.686521 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8632edd-0215-48b6-8ce7-75a287a94cb6 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:48.695936 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51894983-f06e-4f6d-8c7a-616cf9c07ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:48.696780 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51894983-f06e-4f6d-8c7a-616cf9c07ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:48.697184 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51894983-f06e-4f6d-8c7a-616cf9c07ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:48.697527 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51894983-f06e-4f6d-8c7a-616cf9c07ae7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:48.698687 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51894983-f06e-4f6d-8c7a-616cf9c07ae7 None admin] RBAC: Adding query filter params (interface=public, service_id=f276f80b7d1f4281bd5a8a1ae02c3ab2, region_id=RegionOne) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:48.699055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51894983-f06e-4f6d-8c7a-616cf9c07ae7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:48.700917 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51894983-f06e-4f6d-8c7a-616cf9c07ae7 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:48.713066 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 445/907] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Jun 15 19:25:48 2022] GET /identity/v3/endpoints?service_id=f276f80b7d1f4281bd5a8a1ae02c3ab2&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:48.715256 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51894983-f06e-4f6d-8c7a-616cf9c07ae7 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:48.739120 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a464cc43-ae4d-4932-b792-e745e55da390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:48.739732 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464cc43-ae4d-4932-b792-e745e55da390 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:48.740047 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464cc43-ae4d-4932-b792-e745e55da390 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:48.740410 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464cc43-ae4d-4932-b792-e745e55da390 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:48.741415 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a464cc43-ae4d-4932-b792-e745e55da390 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:48.741806 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a464cc43-ae4d-4932-b792-e745e55da390 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:48.743665 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a464cc43-ae4d-4932-b792-e745e55da390 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:48.760275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 463/908] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:48.765961 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a464cc43-ae4d-4932-b792-e745e55da390 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:48.792577 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:48.851553 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51894983-f06e-4f6d-8c7a-616cf9c07ae7 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:48.879813 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a464cc43-ae4d-4932-b792-e745e55da390 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:48.898133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a6d91e-55ed-4c08-a8e7-f8b34e555bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:48.899808 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6d91e-55ed-4c08-a8e7-f8b34e555bcb None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:48.900317 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6d91e-55ed-4c08-a8e7-f8b34e555bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:48.900817 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6d91e-55ed-4c08-a8e7-f8b34e555bcb None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:48.901715 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a6d91e-55ed-4c08-a8e7-f8b34e555bcb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:48.904100 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a6d91e-55ed-4c08-a8e7-f8b34e555bcb None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:48.918284 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 446/909] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Jun 15 19:25:48 2022] GET /identity/v3/services => generated 1497 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:25:48.924359 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4827017-07b1-4ec3-bc06-926dd436a16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:48.925242 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4827017-07b1-4ec3-bc06-926dd436a16a None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:48.925527 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4827017-07b1-4ec3-bc06-926dd436a16a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:48.925805 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4827017-07b1-4ec3-bc06-926dd436a16a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:48.934297 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4827017-07b1-4ec3-bc06-926dd436a16a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:48.935840 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4827017-07b1-4ec3-bc06-926dd436a16a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:48.951917 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-d4827017-07b1-4ec3-bc06-926dd436a16a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:48.951917 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:48.951917 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:48.951917 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:48.951917 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:48.951917 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:48.951917 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:48.951917 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:48.951917 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:48.951917 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:48.951917 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:48.951917 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:48.951917 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 19:25:48.951917 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 19:25:48.951917 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 19:25:48.952919 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 19:25:48.952919 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:48.952919 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:48.952919 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:48.952919 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:48.952919 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:48.952919 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:48.952919 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:48.952919 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:48.952919 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:48.952919 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:48.952919 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:48.952919 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:48.952919 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:48.952919 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:48.952919 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:48.952919 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:48.954308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 19:25:48.954308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 19:25:48.954308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:25:48.954308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:25:48.954308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:25:48.954308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:25:48.954308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:48.954308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:48.954308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 464/910] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:25:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:48.959447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a6d91e-55ed-4c08-a8e7-f8b34e555bcb None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:49.081386 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a6d91e-55ed-4c08-a8e7-f8b34e555bcb None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:49.082830 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4c912-a141-4f69-9ec0-17454c3e21fe None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:49.083123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4c912-a141-4f69-9ec0-17454c3e21fe None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:49.083403 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4c912-a141-4f69-9ec0-17454c3e21fe None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:49.085126 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 465/911] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:49.090647 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907060f8-f37b-48f8-b2b7-bad8697dd13f None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:49.090953 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907060f8-f37b-48f8-b2b7-bad8697dd13f None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:49.091209 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907060f8-f37b-48f8-b2b7-bad8697dd13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:49.110147 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:49.113694 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-907060f8-f37b-48f8-b2b7-bad8697dd13f None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:49.115658 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-907060f8-f37b-48f8-b2b7-bad8697dd13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:49.117598 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:49.138250 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4577ade-5c85-41bd-be43-cef739b84508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:49.139271 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4577ade-5c85-41bd-be43-cef739b84508 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:49.139764 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4577ade-5c85-41bd-be43-cef739b84508 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:49.140254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4577ade-5c85-41bd-be43-cef739b84508 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:49.141248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4577ade-5c85-41bd-be43-cef739b84508 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:49.141759 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4577ade-5c85-41bd-be43-cef739b84508 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:49.143554 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4577ade-5c85-41bd-be43-cef739b84508 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:49.157450 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 447/912] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jun 15 19:25:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:49.163840 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4577ade-5c85-41bd-be43-cef739b84508 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:49.206258 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 466/913] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:25:49.228326 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907060f8-f37b-48f8-b2b7-bad8697dd13f None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:49.278896 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4577ade-5c85-41bd-be43-cef739b84508 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:49.323324 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a91ae-2a26-4f92-a36a-b7a554f9d34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:49.324445 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a91ae-2a26-4f92-a36a-b7a554f9d34e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:49.324730 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a91ae-2a26-4f92-a36a-b7a554f9d34e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:49.325007 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a91ae-2a26-4f92-a36a-b7a554f9d34e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:49.325844 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1a91ae-2a26-4f92-a36a-b7a554f9d34e None admin] RBAC: Adding query filter params (role.id=34c882c657e2470e9bbe75d07694392d, scope.project.id=7fb8cc0eb560465e972ffd45867ee73b, user.id=67aa048d409c46e9a729aec9d1b2286e) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:49.326177 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1a91ae-2a26-4f92-a36a-b7a554f9d34e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:49.331101 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1a91ae-2a26-4f92-a36a-b7a554f9d34e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:49.342665 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 448/914] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:25:49 2022] GET /identity/v3/role_assignments?user.id=67aa048d409c46e9a729aec9d1b2286e&scope.project.id=7fb8cc0eb560465e972ffd45867ee73b&role.id=34c882c657e2470e9bbe75d07694392d => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:49.365549 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907060f8-f37b-48f8-b2b7-bad8697dd13f None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:49.405706 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa645faf-8883-43a5-a7e0-8c1583419804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:49.406569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa645faf-8883-43a5-a7e0-8c1583419804 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:49.406850 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa645faf-8883-43a5-a7e0-8c1583419804 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:49.407125 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa645faf-8883-43a5-a7e0-8c1583419804 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:49.407796 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa645faf-8883-43a5-a7e0-8c1583419804 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:49.409201 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa645faf-8883-43a5-a7e0-8c1583419804 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:49.422182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-aa645faf-8883-43a5-a7e0-8c1583419804 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 19:25:49.422182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:49.422182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 19:25:49.422182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 19:25:49.422182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 19:25:49.422182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 19:25:49.422182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 19:25:49.422182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:25:49.422182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 19:25:49.422182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:49.422182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 19:25:49.422182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:49.422182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:49.422182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:49.422182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:49.422182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:49.422182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:49.423308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:49.423308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:49.423308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:49.423308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:49.423308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:49.423308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:49.423308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 19:25:49.423308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 19:25:49.423308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 19:25:49.423308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 19:25:49.423308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:49.423308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:49.423308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:49.423308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:49.423308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:49.423308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:49.424687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:49.424687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:49.424687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:49.424687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:49.424687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:49.424687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:49.424687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:49.424687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:49.424687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:49.424687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:49.424687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 19:25:49.424687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:25:49.424687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 19:25:49.424687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:49.424687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 467/915] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jun 15 19:25:49 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:25:49.429813 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a91ae-2a26-4f92-a36a-b7a554f9d34e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:49.481714 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:49.541358 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a91ae-2a26-4f92-a36a-b7a554f9d34e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:49.575745 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829edb60-37f4-44e9-a7c0-3a4f4129fa5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:49.576416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829edb60-37f4-44e9-a7c0-3a4f4129fa5a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:49.576698 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829edb60-37f4-44e9-a7c0-3a4f4129fa5a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:49.576977 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829edb60-37f4-44e9-a7c0-3a4f4129fa5a None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:49.577689 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829edb60-37f4-44e9-a7c0-3a4f4129fa5a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:49.577996 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829edb60-37f4-44e9-a7c0-3a4f4129fa5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:49.579599 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829edb60-37f4-44e9-a7c0-3a4f4129fa5a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:49.584785 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 449/916] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jun 15 19:25:49 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:49.590841 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa645faf-8883-43a5-a7e0-8c1583419804 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:49.594400 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:49.692725 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce60a3a-7fb8-4ab4-b422-77657e41101e None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:49.693079 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce60a3a-7fb8-4ab4-b422-77657e41101e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:49.693283 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce60a3a-7fb8-4ab4-b422-77657e41101e None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:49.694942 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 450/917] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:49.700422 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0cb56-131a-46b5-bf99-2ec9fad1b206 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:49.700808 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0cb56-131a-46b5-bf99-2ec9fad1b206 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:49.701143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0cb56-131a-46b5-bf99-2ec9fad1b206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:49.706596 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa645faf-8883-43a5-a7e0-8c1583419804 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:49.719715 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-72f0cb56-131a-46b5-bf99-2ec9fad1b206 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:49.720897 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-72f0cb56-131a-46b5-bf99-2ec9fad1b206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:49.749480 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809572cd-bbbc-4ed3-9b81-a9afaacbf98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:49.751091 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809572cd-bbbc-4ed3-9b81-a9afaacbf98f None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:49.751861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809572cd-bbbc-4ed3-9b81-a9afaacbf98f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:49.752357 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809572cd-bbbc-4ed3-9b81-a9afaacbf98f None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:49.753927 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809572cd-bbbc-4ed3-9b81-a9afaacbf98f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:49.754723 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809572cd-bbbc-4ed3-9b81-a9afaacbf98f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:49.756301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809572cd-bbbc-4ed3-9b81-a9afaacbf98f None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:49.763438 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 468/918] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Jun 15 19:25:49 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:49.771473 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809572cd-bbbc-4ed3-9b81-a9afaacbf98f None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:49.806394 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 451/919] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 19:25:49.824036 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f0cb56-131a-46b5-bf99-2ec9fad1b206 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:49.853981 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:49.872133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:49.903504 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809572cd-bbbc-4ed3-9b81-a9afaacbf98f None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:49.940188 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f0cb56-131a-46b5-bf99-2ec9fad1b206 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:49.949773 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a4ddb-8cbb-47cb-93b0-1534124db783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:49.951011 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a4ddb-8cbb-47cb-93b0-1534124db783 None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:49.951419 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a4ddb-8cbb-47cb-93b0-1534124db783 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:49.951794 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a4ddb-8cbb-47cb-93b0-1534124db783 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:49.953172 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276a4ddb-8cbb-47cb-93b0-1534124db783 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:49.955063 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276a4ddb-8cbb-47cb-93b0-1534124db783 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:49.979078 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0fb1b7-06b9-4999-844f-ab15350af6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:49.980100 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0fb1b7-06b9-4999-844f-ab15350af6ba None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:49.980657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0fb1b7-06b9-4999-844f-ab15350af6ba None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:49.981183 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0fb1b7-06b9-4999-844f-ab15350af6ba None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:49.982060 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0fb1b7-06b9-4999-844f-ab15350af6ba None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:49.983826 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0fb1b7-06b9-4999-844f-ab15350af6ba None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:49.986681 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 469/920] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Jun 15 19:25:49 2022] POST /identity/v3/endpoints => generated 346 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:25:49.999994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-fe0fb1b7-06b9-4999-844f-ab15350af6ba None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 19:25:49.999994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:49.999994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 19:25:49.999994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 19:25:49.999994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 19:25:49.999994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 19:25:49.999994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 19:25:49.999994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:25:49.999994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 19:25:49.999994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:49.999994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 19:25:49.999994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:49.999994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:49.999994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:49.999994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:49.999994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:49.999994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:50.001086 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:50.001086 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:50.001086 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:50.001086 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:50.001086 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:50.001086 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:50.001086 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 19:25:50.001086 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 19:25:50.001086 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 19:25:50.001086 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 19:25:50.001086 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:50.001086 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:50.001086 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:50.001086 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:50.001086 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:50.001086 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:50.001954 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:50.001954 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:50.001954 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:50.001954 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:50.001954 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:50.001954 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:50.001954 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:50.001954 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:50.001954 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:50.001954 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:50.001954 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 19:25:50.001954 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:25:50.001954 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 19:25:50.001954 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:50.005132 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 452/921] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Jun 15 19:25:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:50.011966 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a4ddb-8cbb-47cb-93b0-1534124db783 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:50.134719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a4ddb-8cbb-47cb-93b0-1534124db783 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:50.159408 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:50.180929 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa17e0e4-7ebc-438b-978e-d82b3b824188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:50.182010 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa17e0e4-7ebc-438b-978e-d82b3b824188 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:50.182637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa17e0e4-7ebc-438b-978e-d82b3b824188 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:50.182918 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa17e0e4-7ebc-438b-978e-d82b3b824188 None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:50.183890 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa17e0e4-7ebc-438b-978e-d82b3b824188 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:50.184605 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa17e0e4-7ebc-438b-978e-d82b3b824188 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:50.185985 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa17e0e4-7ebc-438b-978e-d82b3b824188 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:50.191334 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 470/922] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Jun 15 19:25:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:50.197647 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0fb1b7-06b9-4999-844f-ab15350af6ba None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:50.304459 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0fb1b7-06b9-4999-844f-ab15350af6ba None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:50.306697 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:50.341704 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f8d063-cba7-4e51-93e9-95d51655af7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:50.342415 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8d063-cba7-4e51-93e9-95d51655af7d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:50.342700 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8d063-cba7-4e51-93e9-95d51655af7d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:50.342973 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8d063-cba7-4e51-93e9-95d51655af7d None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:50.343680 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f8d063-cba7-4e51-93e9-95d51655af7d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:50.343990 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f8d063-cba7-4e51-93e9-95d51655af7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:50.345354 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f8d063-cba7-4e51-93e9-95d51655af7d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:50.350834 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 453/923] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Jun 15 19:25:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:25:50.356977 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa17e0e4-7ebc-438b-978e-d82b3b824188 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:50.458645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa17e0e4-7ebc-438b-978e-d82b3b824188 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:50.496410 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a71b55a-a5c0-4003-8cbd-9204daada270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:50.497443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a71b55a-a5c0-4003-8cbd-9204daada270 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:50.497949 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a71b55a-a5c0-4003-8cbd-9204daada270 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:50.498549 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a71b55a-a5c0-4003-8cbd-9204daada270 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:50.499540 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a71b55a-a5c0-4003-8cbd-9204daada270 None admin] RBAC: Adding query filter params (interface=public, service_id=7c4ce0dfa8c0498784b7c3b8d40ddbb3, region_id=RegionOne) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:50.500054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a71b55a-a5c0-4003-8cbd-9204daada270 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:50.501613 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a71b55a-a5c0-4003-8cbd-9204daada270 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:50.507512 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 471/924] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Jun 15 19:25:50 2022] GET /identity/v3/endpoints?service_id=7c4ce0dfa8c0498784b7c3b8d40ddbb3&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:50.514285 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f8d063-cba7-4e51-93e9-95d51655af7d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:50.619198 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f8d063-cba7-4e51-93e9-95d51655af7d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:50.654190 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083f7b10-6788-4ba9-9574-d3a96348fccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:50.655214 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f7b10-6788-4ba9-9574-d3a96348fccb None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:50.655702 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f7b10-6788-4ba9-9574-d3a96348fccb None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:50.656210 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f7b10-6788-4ba9-9574-d3a96348fccb None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:50.657104 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083f7b10-6788-4ba9-9574-d3a96348fccb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:50.658711 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083f7b10-6788-4ba9-9574-d3a96348fccb None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:50.665832 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 454/925] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Jun 15 19:25:50 2022] GET /identity/v3/services => generated 1497 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:25:51.198984 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfc1f3-7994-41d9-be9a-cc35a5eb8726 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:51.199536 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfc1f3-7994-41d9-be9a-cc35a5eb8726 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:51.199536 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfc1f3-7994-41d9-be9a-cc35a5eb8726 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:51.201253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 472/926] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:51.206470 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c87c5-e5fc-4d60-9364-086530f3ff94 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:51.206765 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c87c5-e5fc-4d60-9364-086530f3ff94 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:51.207045 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c87c5-e5fc-4d60-9364-086530f3ff94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:51.224685 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-f41c87c5-e5fc-4d60-9364-086530f3ff94 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:51.225906 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f41c87c5-e5fc-4d60-9364-086530f3ff94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:51.252235 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:51.297830 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 455/927] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:25:51.319908 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bfc1f3-7994-41d9-be9a-cc35a5eb8726 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:51.371499 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:51.428724 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bfc1f3-7994-41d9-be9a-cc35a5eb8726 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:51.465250 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aea810-6746-4d34-8090-1d9b9914f5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:51.466003 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aea810-6746-4d34-8090-1d9b9914f5ce None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:51.466316 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aea810-6746-4d34-8090-1d9b9914f5ce None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:51.466591 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aea810-6746-4d34-8090-1d9b9914f5ce None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:51.467258 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aea810-6746-4d34-8090-1d9b9914f5ce None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:51.468839 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aea810-6746-4d34-8090-1d9b9914f5ce None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:51.482477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-71aea810-6746-4d34-8090-1d9b9914f5ce None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 19:25:51.482477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:51.482477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 19:25:51.482477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 19:25:51.482477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 19:25:51.482477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 19:25:51.482477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 19:25:51.482477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:25:51.482477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 19:25:51.482477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:51.482477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 19:25:51.482477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:51.482477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:51.482477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:51.482477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:51.482477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:51.483552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:51.483552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:51.483552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:51.483552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:51.483552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:51.483552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:51.483552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:51.483552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 19:25:51.483552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 19:25:51.483552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 19:25:51.483552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 19:25:51.483552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:51.483552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:51.483552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:51.483552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:51.483552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:51.484397 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:51.484397 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:51.484397 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:51.484397 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:51.484397 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:51.484397 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:51.484397 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:51.484397 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:51.484397 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:51.484397 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:51.484397 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:51.484397 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 19:25:51.484397 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:25:51.484397 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 19:25:51.484397 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:51.484397 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 473/928] 10.222.0.15 () {56 vars in 1132 bytes} [Wed Jun 15 19:25:51 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:51.490039 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c87c5-e5fc-4d60-9364-086530f3ff94 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:51.521909 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e20524-15b7-43e8-a003-f36a8a7539f0 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:51.522209 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e20524-15b7-43e8-a003-f36a8a7539f0 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:51.522489 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e20524-15b7-43e8-a003-f36a8a7539f0 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:51.524934 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 474/929] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 19:25:51.529907 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40163f4d-0c77-4e53-8fde-45b333155773 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:51.530025 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40163f4d-0c77-4e53-8fde-45b333155773 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:51.530274 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40163f4d-0c77-4e53-8fde-45b333155773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:51.546941 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-40163f4d-0c77-4e53-8fde-45b333155773 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:51.548183 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-40163f4d-0c77-4e53-8fde-45b333155773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:51.604227 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c87c5-e5fc-4d60-9364-086530f3ff94 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:51.622426 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 475/930] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 19:25:51.640457 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40163f4d-0c77-4e53-8fde-45b333155773 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:51.641599 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e33cb0a-83b1-4cea-9ed2-7033e3a80caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:51.642629 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33cb0a-83b1-4cea-9ed2-7033e3a80caa None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:51.643106 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33cb0a-83b1-4cea-9ed2-7033e3a80caa None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:51.643633 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33cb0a-83b1-4cea-9ed2-7033e3a80caa None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:51.644523 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e33cb0a-83b1-4cea-9ed2-7033e3a80caa None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:51.645017 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e33cb0a-83b1-4cea-9ed2-7033e3a80caa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:51.646596 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e33cb0a-83b1-4cea-9ed2-7033e3a80caa None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:51.652870 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 456/931] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Jun 15 19:25:51 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:51.658798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e33cb0a-83b1-4cea-9ed2-7033e3a80caa None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:51.690554 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:51.699941 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:51.777696 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40163f4d-0c77-4e53-8fde-45b333155773 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:51.792663 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e33cb0a-83b1-4cea-9ed2-7033e3a80caa None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:51.827124 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c800d80-1e29-428c-a91e-c970b1845c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:51.828582 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c800d80-1e29-428c-a91e-c970b1845c89 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:51.829027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6071e1-d7c5-49c1-8f70-82dbd5b29b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:51.829315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c800d80-1e29-428c-a91e-c970b1845c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:51.829687 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6071e1-d7c5-49c1-8f70-82dbd5b29b9e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:51.829967 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6071e1-d7c5-49c1-8f70-82dbd5b29b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:51.830272 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6071e1-d7c5-49c1-8f70-82dbd5b29b9e None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:51.830374 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c800d80-1e29-428c-a91e-c970b1845c89 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:51.830980 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6071e1-d7c5-49c1-8f70-82dbd5b29b9e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:51.831281 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6071e1-d7c5-49c1-8f70-82dbd5b29b9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:51.832649 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6071e1-d7c5-49c1-8f70-82dbd5b29b9e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:51.842500 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 457/932] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Jun 15 19:25:51 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:51.843721 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6071e1-d7c5-49c1-8f70-82dbd5b29b9e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:51.850654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c800d80-1e29-428c-a91e-c970b1845c89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:25:51.863159 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-7c800d80-1e29-428c-a91e-c970b1845c89 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 19:25:51.863159 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:51.863159 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:51.863159 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:51.863159 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:51.863159 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:51.863159 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:51.863159 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:51.863159 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:51.863159 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:51.863159 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:51.863159 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:51.863159 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:25:51.863159 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:25:51.863159 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:25:51.863159 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:25:51.864123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:25:51.864123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:25:51.864123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:25:51.864123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:25:51.864123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:51.864123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:51.864123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:51.864123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:51.864123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:51.864123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:51.864123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:51.864123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:51.864123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:51.864123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:51.864123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:51.864123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:51.864963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:51.864963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:51.864963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:51.864963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:51.864963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:25:51.864963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:25:51.864963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:25:51.864963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:25:51.864963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:25:51.864963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:25:51.864963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 19:25:51.864963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:51.873044 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 476/933] 10.222.0.15 () {56 vars in 1120 bytes} [Wed Jun 15 19:25:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:25:51.873737 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c800d80-1e29-428c-a91e-c970b1845c89 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:51.966504 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:51.977961 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:51.981122 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c800d80-1e29-428c-a91e-c970b1845c89 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:51.985644 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6071e1-d7c5-49c1-8f70-82dbd5b29b9e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:52.018336 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59347f0-32e6-4d7b-bab2-0a4ac817a4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:52.019252 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59347f0-32e6-4d7b-bab2-0a4ac817a4b9 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:52.019897 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59347f0-32e6-4d7b-bab2-0a4ac817a4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:52.020086 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59347f0-32e6-4d7b-bab2-0a4ac817a4b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:52.020940 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59347f0-32e6-4d7b-bab2-0a4ac817a4b9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:52.021396 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59347f0-32e6-4d7b-bab2-0a4ac817a4b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:52.023432 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59347f0-32e6-4d7b-bab2-0a4ac817a4b9 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:52.031127 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85861653-c617-4bb2-b684-d3a03ac94ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:52.032007 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85861653-c617-4bb2-b684-d3a03ac94ec8 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:52.032437 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85861653-c617-4bb2-b684-d3a03ac94ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:52.032853 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85861653-c617-4bb2-b684-d3a03ac94ec8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:52.037850 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 477/934] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Jun 15 19:25:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:52.037996 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85861653-c617-4bb2-b684-d3a03ac94ec8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:52.037996 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85861653-c617-4bb2-b684-d3a03ac94ec8 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:52.038913 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59347f0-32e6-4d7b-bab2-0a4ac817a4b9 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:52.068276 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 458/935] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Jun 15 19:25:51 2022] POST /identity/v3/endpoints => generated 347 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:25:52.166602 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59347f0-32e6-4d7b-bab2-0a4ac817a4b9 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:52.215258 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24bc693-de92-45d1-8594-1668c3c30667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:52.216926 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24bc693-de92-45d1-8594-1668c3c30667 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:52.217427 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24bc693-de92-45d1-8594-1668c3c30667 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:52.217901 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24bc693-de92-45d1-8594-1668c3c30667 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:52.231424 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24bc693-de92-45d1-8594-1668c3c30667 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:52.233347 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24bc693-de92-45d1-8594-1668c3c30667 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:52.255654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-a24bc693-de92-45d1-8594-1668c3c30667 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:52.255654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:52.255654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:52.255654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:52.255654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:52.255654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:52.255654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:52.255654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:52.255654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:52.255654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:52.255654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:52.255654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:52.255654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:52.255654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:52.255654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:52.256612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:52.256612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:52.256612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:52.256612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:52.256612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:52.256612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:52.256612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:52.256612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:52.256612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:52.256612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:52.256612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:52.256612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:52.256612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:52.256612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:52.256612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:52.256612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:52.256612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:52.257585 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:52.257585 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:52.257585 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:52.257585 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:52.257585 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 478/936] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:25:52.265046 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85861653-c617-4bb2-b684-d3a03ac94ec8 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:52.300400 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ddc3b-180b-4341-93f7-e5e185d002a8 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:52.300704 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ddc3b-180b-4341-93f7-e5e185d002a8 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:52.300873 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ddc3b-180b-4341-93f7-e5e185d002a8 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:52.302937 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 479/937] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 19:25:52.308797 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f018a93-facb-43aa-8d8c-1c0db8cd41cb None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:52.309086 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f018a93-facb-43aa-8d8c-1c0db8cd41cb None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:52.309363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f018a93-facb-43aa-8d8c-1c0db8cd41cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:52.336417 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-4f018a93-facb-43aa-8d8c-1c0db8cd41cb None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:52.338290 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-4f018a93-facb-43aa-8d8c-1c0db8cd41cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:52.348204 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:52.398765 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85861653-c617-4bb2-b684-d3a03ac94ec8 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:52.418582 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 480/938] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:25:52.441808 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba1b9c-a152-4c5e-9181-9d2cc9e77899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:52.442241 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f018a93-facb-43aa-8d8c-1c0db8cd41cb None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:52.443029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba1b9c-a152-4c5e-9181-9d2cc9e77899 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:52.443493 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba1b9c-a152-4c5e-9181-9d2cc9e77899 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:52.443958 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba1b9c-a152-4c5e-9181-9d2cc9e77899 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:52.445170 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ba1b9c-a152-4c5e-9181-9d2cc9e77899 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:52.446412 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ba1b9c-a152-4c5e-9181-9d2cc9e77899 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:52.449605 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ba1b9c-a152-4c5e-9181-9d2cc9e77899 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:52.463301 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 459/939] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:52.468319 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba1b9c-a152-4c5e-9181-9d2cc9e77899 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:52.472252 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:52.553143 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f018a93-facb-43aa-8d8c-1c0db8cd41cb None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:52.576024 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba1b9c-a152-4c5e-9181-9d2cc9e77899 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:52.589408 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c4eee7-4bd6-4f24-9ef9-1e7a6c95abb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:52.590075 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4eee7-4bd6-4f24-9ef9-1e7a6c95abb2 None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:52.590378 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4eee7-4bd6-4f24-9ef9-1e7a6c95abb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:52.590655 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4eee7-4bd6-4f24-9ef9-1e7a6c95abb2 None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:52.591432 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c4eee7-4bd6-4f24-9ef9-1e7a6c95abb2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:52.592772 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c4eee7-4bd6-4f24-9ef9-1e7a6c95abb2 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:52.594161 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: WARNING keystone.server.flask.common [None req-b8c4eee7-4bd6-4f24-9ef9-1e7a6c95abb2 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 15 19:25:52.604234 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:52.614146 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44443c-e50e-48a7-b57e-9e2a149b70f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:52.614664 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.sql.core [None req-b8c4eee7-4bd6-4f24-9ef9-1e7a6c95abb2 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 15 19:25:52.614664 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 15 19:25:52.614664 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [parameters: {'id': 'aa4d5e80dba14394a1719a7229955c89', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 15 19:25:52.614664 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80223) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 15 19:25:52.615888 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44443c-e50e-48a7-b57e-9e2a149b70f2 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:52.616379 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44443c-e50e-48a7-b57e-9e2a149b70f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:52.616802 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44443c-e50e-48a7-b57e-9e2a149b70f2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:52.622742 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-b8c4eee7-4bd6-4f24-9ef9-1e7a6c95abb2 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id aa4d5e80dba14394a1719a7229955c89.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id aa4d5e80dba14394a1719a7229955c89. Jun 15 19:25:52.622742 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:52.622742 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 19:25:52.622742 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 19:25:52.622742 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 19:25:52.622742 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 19:25:52.622742 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 19:25:52.622742 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 19:25:52.622742 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 19:25:52.622742 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application conn.query(q) Jun 15 19:25:52.622742 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 19:25:52.622742 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 19:25:52.622742 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 19:25:52.623737 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application result.read() Jun 15 19:25:52.623737 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 19:25:52.623737 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 19:25:52.623737 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 19:25:52.623737 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 19:25:52.623737 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 19:25:52.623737 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 19:25:52.623737 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 19:25:52.623737 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 19:25:52.623737 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 15 19:25:52.623737 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:52.623737 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 15 19:25:52.623737 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:52.623737 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:52.623737 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 15 19:25:52.623737 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 15 19:25:52.623737 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 15 19:25:52.623737 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 15 19:25:52.624928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 19:25:52.624928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application next(self.gen) Jun 15 19:25:52.624928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 15 19:25:52.624928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application yield resource Jun 15 19:25:52.624928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 19:25:52.624928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application next(self.gen) Jun 15 19:25:52.624928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 15 19:25:52.624928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self.session.rollback() Jun 15 19:25:52.624928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 15 19:25:52.624928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self.force_reraise() Jun 15 19:25:52.624928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 15 19:25:52.624928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise self.value Jun 15 19:25:52.624928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 15 19:25:52.624928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 15 19:25:52.624928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 15 19:25:52.624928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application session.commit() Jun 15 19:25:52.624928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 15 19:25:52.626035 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 15 19:25:52.626035 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 15 19:25:52.626035 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._prepare_impl() Jun 15 19:25:52.626035 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 15 19:25:52.626035 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self.session.flush() Jun 15 19:25:52.626035 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 15 19:25:52.626035 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._flush(objects) Jun 15 19:25:52.626035 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 15 19:25:52.626035 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 15 19:25:52.626035 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 15 19:25:52.626035 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application compat.raise_( Jun 15 19:25:52.626035 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 19:25:52.626035 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception Jun 15 19:25:52.626035 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 15 19:25:52.626035 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application flush_context.execute() Jun 15 19:25:52.626035 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 15 19:25:52.626035 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rec.execute(self) Jun 15 19:25:52.627185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 15 19:25:52.627185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 15 19:25:52.627185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 15 19:25:52.627185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 15 19:25:52.627185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 15 19:25:52.627185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 15 19:25:52.627185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 15 19:25:52.627185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 15 19:25:52.627185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 15 19:25:52.627185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 15 19:25:52.627185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 15 19:25:52.627185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 15 19:25:52.627185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 15 19:25:52.627185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 15 19:25:52.627185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 15 19:25:52.628146 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 15 19:25:52.628146 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 19:25:52.628146 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception Jun 15 19:25:52.628146 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 19:25:52.628146 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 19:25:52.628146 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 19:25:52.628146 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 19:25:52.628146 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 19:25:52.628146 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 19:25:52.628146 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 19:25:52.628146 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application conn.query(q) Jun 15 19:25:52.628146 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 19:25:52.628146 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 19:25:52.628146 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 19:25:52.628146 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application result.read() Jun 15 19:25:52.628146 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 19:25:52.628146 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 19:25:52.629179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 19:25:52.629179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 19:25:52.629179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 19:25:52.629179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 19:25:52.629179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 19:25:52.629179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 19:25:52.629179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 15 19:25:52.629179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 15 19:25:52.629179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [parameters: {'id': 'aa4d5e80dba14394a1719a7229955c89', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 15 19:25:52.629179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 15 19:25:52.629179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:52.629179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 19:25:52.629179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:52.629179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:52.629179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 15 19:25:52.630776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 15 19:25:52.630776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 15 19:25:52.630776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 15 19:25:52.630776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 15 19:25:52.630776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:52.630776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 19:25:52.630776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:52.630776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:52.630776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:52.630776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:52.630776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:52.630776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:52.630776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:52.630776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:52.630776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:52.630776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:52.630776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:52.632665 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:52.632665 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 15 19:25:52.632665 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 15 19:25:52.632665 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:52.632665 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:52.632665 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 15 19:25:52.632665 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 15 19:25:52.632665 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id aa4d5e80dba14394a1719a7229955c89. Jun 15 19:25:52.632665 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:52.632665 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 481/940] 10.222.0.15 () {60 vars in 1140 bytes} [Wed Jun 15 19:25:52 2022] POST /identity/v3/projects => generated 262 bytes in 183 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:25:52.632665 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c4eee7-4bd6-4f24-9ef9-1e7a6c95abb2 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:52.637491 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec44443c-e50e-48a7-b57e-9e2a149b70f2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:52.639179 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec44443c-e50e-48a7-b57e-9e2a149b70f2 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:52.655821 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-ec44443c-e50e-48a7-b57e-9e2a149b70f2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 19:25:52.655821 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:52.655821 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:52.655821 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:52.655821 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:52.655821 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:52.655821 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:52.655821 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:52.655821 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:52.655821 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:52.655821 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:52.655821 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:52.655821 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 19:25:52.655821 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 19:25:52.655821 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 19:25:52.655821 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 19:25:52.656833 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:52.656833 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:52.656833 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:25:52.656833 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:52.656833 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:25:52.656833 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:52.656833 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:52.656833 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:52.656833 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:52.656833 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:52.656833 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:52.656833 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:52.656833 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:52.656833 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:52.656833 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:52.656833 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:52.656833 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:52.656833 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:52.657836 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:52.657836 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:52.657836 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 19:25:52.657836 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 19:25:52.657836 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 19:25:52.657836 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 19:25:52.657836 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 19:25:52.657836 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 19:25:52.657836 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 19:25:52.657836 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:52.660389 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 460/941] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Jun 15 19:25:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:25:52.666189 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44443c-e50e-48a7-b57e-9e2a149b70f2 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:52.732374 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:52.744917 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c4eee7-4bd6-4f24-9ef9-1e7a6c95abb2 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:52.774576 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44443c-e50e-48a7-b57e-9e2a149b70f2 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:52.781182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853484d-82a7-4724-a2e7-f0b3db2f8038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:52.782120 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853484d-82a7-4724-a2e7-f0b3db2f8038 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:52.782495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853484d-82a7-4724-a2e7-f0b3db2f8038 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:52.782839 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853484d-82a7-4724-a2e7-f0b3db2f8038 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:52.791393 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853484d-82a7-4724-a2e7-f0b3db2f8038 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:52.793208 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853484d-82a7-4724-a2e7-f0b3db2f8038 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:52.808501 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-1853484d-82a7-4724-a2e7-f0b3db2f8038 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:52.808501 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:52.808501 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:52.808501 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:52.808501 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:52.808501 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:52.808501 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:52.808501 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:52.808501 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:52.808501 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:52.808501 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:52.808501 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:52.808501 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 19:25:52.808501 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 19:25:52.808501 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 19:25:52.809432 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 19:25:52.809432 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:52.809432 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:52.809432 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:52.809432 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:52.809432 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:52.809432 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:52.809432 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:52.809432 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:52.809432 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:52.809432 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:52.809432 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:52.809432 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:52.809432 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:52.809432 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:52.809432 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:52.809432 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:52.810362 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 19:25:52.810362 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 19:25:52.810362 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:25:52.810362 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:25:52.810362 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:25:52.810362 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:25:52.810362 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:52.810362 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:52.810362 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 482/942] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:25:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:52.815326 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853484d-82a7-4724-a2e7-f0b3db2f8038 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:52.820156 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12918880-d961-47be-9f2a-53c44ab494bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:52.821753 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12918880-d961-47be-9f2a-53c44ab494bc None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:52.822281 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12918880-d961-47be-9f2a-53c44ab494bc None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:52.822864 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12918880-d961-47be-9f2a-53c44ab494bc None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:52.823991 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12918880-d961-47be-9f2a-53c44ab494bc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:52.824534 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12918880-d961-47be-9f2a-53c44ab494bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:52.826242 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12918880-d961-47be-9f2a-53c44ab494bc None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:52.845971 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 461/943] 10.222.0.15 () {56 vars in 1149 bytes} [Wed Jun 15 19:25:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:52.851441 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12918880-d961-47be-9f2a-53c44ab494bc None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:52.880260 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:52.929977 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853484d-82a7-4724-a2e7-f0b3db2f8038 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:52.956055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12918880-d961-47be-9f2a-53c44ab494bc None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:52.968834 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8dc18-67d7-44ec-a925-524a81008476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:52.969497 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8dc18-67d7-44ec-a925-524a81008476 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:52.969784 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8dc18-67d7-44ec-a925-524a81008476 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:52.970061 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8dc18-67d7-44ec-a925-524a81008476 None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:52.970842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf8dc18-67d7-44ec-a925-524a81008476 None admin] RBAC: Adding query filter params (name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:52.971149 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf8dc18-67d7-44ec-a925-524a81008476 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:52.972593 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf8dc18-67d7-44ec-a925-524a81008476 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:52.980629 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:52.993554 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c6483-ad4f-4306-a347-f5def1b75906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:52.994372 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c6483-ad4f-4306-a347-f5def1b75906 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:52.994654 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c6483-ad4f-4306-a347-f5def1b75906 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:52.994923 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c6483-ad4f-4306-a347-f5def1b75906 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:53.002665 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 483/944] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Jun 15 19:25:52 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:53.003954 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe5745-f191-4565-858a-2f535f49092c None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:53.004242 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe5745-f191-4565-858a-2f535f49092c None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:53.004515 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe5745-f191-4565-858a-2f535f49092c None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:53.006267 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 484/945] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:53.018129 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c6483-ad4f-4306-a347-f5def1b75906 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:53.018353 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7c216-63de-4f9f-b404-bca9f0353d6d None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:53.018353 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7c216-63de-4f9f-b404-bca9f0353d6d None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:53.018353 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7c216-63de-4f9f-b404-bca9f0353d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:53.018618 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c6483-ad4f-4306-a347-f5def1b75906 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:53.036101 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-c5c7c216-63de-4f9f-b404-bca9f0353d6d None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:53.037777 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-c5c7c216-63de-4f9f-b404-bca9f0353d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:53.038892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-040c6483-ad4f-4306-a347-f5def1b75906 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:53.038892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:53.038892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:53.038892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:53.038892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:53.038892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:53.038892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:53.038892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:53.038892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:53.038892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:53.038892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:53.038892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:53.038892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:53.038892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:53.038892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:53.040475 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:53.040475 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:53.040475 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:53.040475 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:53.040475 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:53.040475 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:53.040475 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:53.040475 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:53.040475 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:53.040475 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:53.040475 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:53.040475 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:53.040475 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:53.040475 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:53.040475 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:53.040475 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:53.040475 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:53.041419 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:53.041419 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:53.041419 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:53.041419 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:53.043812 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 462/946] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:53.048896 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c6483-ad4f-4306-a347-f5def1b75906 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:53.128945 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 485/947] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 19:25:53.152714 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c7c216-63de-4f9f-b404-bca9f0353d6d None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:53.183266 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c6483-ad4f-4306-a347-f5def1b75906 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:53.217535 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:53.236743 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c9471-2ef6-4af8-8ca8-d2db7f728824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:53.237490 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c9471-2ef6-4af8-8ca8-d2db7f728824 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:53.237840 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c9471-2ef6-4af8-8ca8-d2db7f728824 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:53.239119 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c9471-2ef6-4af8-8ca8-d2db7f728824 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:53.239119 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328c9471-2ef6-4af8-8ca8-d2db7f728824 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:53.239480 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328c9471-2ef6-4af8-8ca8-d2db7f728824 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:53.240965 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328c9471-2ef6-4af8-8ca8-d2db7f728824 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:53.259236 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 463/948] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:25:53.265190 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c9471-2ef6-4af8-8ca8-d2db7f728824 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:53.300824 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c7c216-63de-4f9f-b404-bca9f0353d6d None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:53.320719 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:53.354621 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff96d2-c124-419c-80f8-7067fee59fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:53.355665 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff96d2-c124-419c-80f8-7067fee59fcb None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:53.356007 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff96d2-c124-419c-80f8-7067fee59fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:53.356297 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff96d2-c124-419c-80f8-7067fee59fcb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:53.357029 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baff96d2-c124-419c-80f8-7067fee59fcb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:53.358729 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baff96d2-c124-419c-80f8-7067fee59fcb None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:53.381017 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-baff96d2-c124-419c-80f8-7067fee59fcb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 19:25:53.381017 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:53.381017 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 19:25:53.381017 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 19:25:53.381017 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 19:25:53.381017 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 19:25:53.381017 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 19:25:53.381017 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:25:53.381017 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 19:25:53.381017 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:53.381017 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 19:25:53.381017 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:53.381017 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:53.381017 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:53.381017 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:53.381017 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:53.381017 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:53.382308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:53.382308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:53.382308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:53.382308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:53.382308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:53.382308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:53.382308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 19:25:53.382308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 19:25:53.382308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 19:25:53.382308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 19:25:53.382308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:53.382308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:53.382308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:53.382308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:53.382308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:53.382308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:53.384149 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:53.384149 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:53.384149 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:53.384149 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:53.384149 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:53.384149 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:53.384149 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:53.384149 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:53.384149 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:53.384149 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:53.384149 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 19:25:53.384149 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:25:53.384149 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 19:25:53.384149 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:53.390918 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 486/949] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Jun 15 19:25:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:25:53.396399 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff96d2-c124-419c-80f8-7067fee59fcb None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:53.408459 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c9471-2ef6-4af8-8ca8-d2db7f728824 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:53.447008 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f79b6-d547-4e3d-9928-5b316943f9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:53.448071 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f79b6-d547-4e3d-9928-5b316943f9e6 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:53.448562 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f79b6-d547-4e3d-9928-5b316943f9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:53.448998 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f79b6-d547-4e3d-9928-5b316943f9e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:53.456605 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9f79b6-d547-4e3d-9928-5b316943f9e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:53.458368 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9f79b6-d547-4e3d-9928-5b316943f9e6 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:53.474689 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-bb9f79b6-d547-4e3d-9928-5b316943f9e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:53.474689 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:53.474689 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:53.474689 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:53.474689 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:53.474689 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:53.474689 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:53.474689 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:53.474689 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:53.474689 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:53.474689 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:53.474689 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:53.474689 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 19:25:53.474689 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 19:25:53.474689 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 19:25:53.475839 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 19:25:53.475839 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:53.475839 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:53.475839 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:53.475839 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:53.475839 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:53.475839 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:53.475839 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:53.475839 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:53.475839 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:53.475839 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:53.475839 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:53.475839 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:53.475839 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:53.475839 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:53.475839 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:53.475839 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:53.476758 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 19:25:53.476758 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 19:25:53.476758 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:25:53.476758 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:25:53.476758 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:25:53.476758 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:25:53.476758 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:53.476758 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:53.484198 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 464/950] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:25:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:53.486872 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f79b6-d547-4e3d-9928-5b316943f9e6 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:53.534069 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff96d2-c124-419c-80f8-7067fee59fcb None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:53.546578 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:53.575261 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a6d348-b2fd-4c3a-8765-1852e073c5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:53.576579 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6d348-b2fd-4c3a-8765-1852e073c5e7 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:53.577032 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6d348-b2fd-4c3a-8765-1852e073c5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:53.577386 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6d348-b2fd-4c3a-8765-1852e073c5e7 None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:53.578761 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a6d348-b2fd-4c3a-8765-1852e073c5e7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:53.579463 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a6d348-b2fd-4c3a-8765-1852e073c5e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:53.582059 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a6d348-b2fd-4c3a-8765-1852e073c5e7 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:53.591602 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 487/951] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Jun 15 19:25:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:25:53.592643 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a6d348-b2fd-4c3a-8765-1852e073c5e7 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:53.594154 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:53.601367 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f79b6-d547-4e3d-9928-5b316943f9e6 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:53.645946 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a089f9-36e0-4c97-9983-ebc6521e58ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:53.647517 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a089f9-36e0-4c97-9983-ebc6521e58ff None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:53.647983 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a089f9-36e0-4c97-9983-ebc6521e58ff None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:53.648685 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a089f9-36e0-4c97-9983-ebc6521e58ff None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:53.650301 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a089f9-36e0-4c97-9983-ebc6521e58ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:53.650858 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a089f9-36e0-4c97-9983-ebc6521e58ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:53.652567 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a089f9-36e0-4c97-9983-ebc6521e58ff None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:53.671314 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 465/952] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jun 15 19:25:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:53.672469 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a089f9-36e0-4c97-9983-ebc6521e58ff None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:53.728486 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a6d348-b2fd-4c3a-8765-1852e073c5e7 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:53.772135 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4f70e-6ec8-483b-a8b9-616762bf1dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:53.773222 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4f70e-6ec8-483b-a8b9-616762bf1dab None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:53.774141 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4f70e-6ec8-483b-a8b9-616762bf1dab None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:53.774593 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4f70e-6ec8-483b-a8b9-616762bf1dab None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:53.775936 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db4f70e-6ec8-483b-a8b9-616762bf1dab None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:53.776388 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db4f70e-6ec8-483b-a8b9-616762bf1dab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:53.778815 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db4f70e-6ec8-483b-a8b9-616762bf1dab None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:53.784838 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a089f9-36e0-4c97-9983-ebc6521e58ff None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:53.786173 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 488/953] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Jun 15 19:25:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:53.791848 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4f70e-6ec8-483b-a8b9-616762bf1dab None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:53.822517 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d4b16-73ca-4e24-ac13-54ce76cc4126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:53.823752 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d4b16-73ca-4e24-ac13-54ce76cc4126 None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:53.824213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d4b16-73ca-4e24-ac13-54ce76cc4126 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:53.824675 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d4b16-73ca-4e24-ac13-54ce76cc4126 None admin] PATH_INFO: `/v3/projects/7fb8cc0eb560465e972ffd45867ee73b/users/67aa048d409c46e9a729aec9d1b2286e/roles/34c882c657e2470e9bbe75d07694392d` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:53.827443 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2d4b16-73ca-4e24-ac13-54ce76cc4126 None admin] RBAC: Authorizing `identity:create_grant(project_id=7fb8cc0eb560465e972ffd45867ee73b, user_id=67aa048d409c46e9a729aec9d1b2286e, role_id=34c882c657e2470e9bbe75d07694392d)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:53.829081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2d4b16-73ca-4e24-ac13-54ce76cc4126 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:53.846136 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:53.854704 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 466/954] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jun 15 19:25:53 2022] PUT /identity/v3/projects/7fb8cc0eb560465e972ffd45867ee73b/users/67aa048d409c46e9a729aec9d1b2286e/roles/34c882c657e2470e9bbe75d07694392d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:25:53.950006 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4f70e-6ec8-483b-a8b9-616762bf1dab None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:53.999282 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8597c5-e6d5-42aa-8034-d821b8454536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:54.000331 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8597c5-e6d5-42aa-8034-d821b8454536 None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:54.000817 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8597c5-e6d5-42aa-8034-d821b8454536 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:54.001293 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8597c5-e6d5-42aa-8034-d821b8454536 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:54.002309 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8597c5-e6d5-42aa-8034-d821b8454536 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:54.003969 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8597c5-e6d5-42aa-8034-d821b8454536 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:54.030077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 489/955] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Jun 15 19:25:53 2022] POST /identity/v3/endpoints => generated 349 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:25:54.361270 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7318feaa-0ee1-4437-a813-ff043d7a9294 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:54.361586 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7318feaa-0ee1-4437-a813-ff043d7a9294 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:54.361837 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7318feaa-0ee1-4437-a813-ff043d7a9294 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:54.363489 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 467/956] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:54.369528 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927654be-26a6-457f-b201-72fdf287b45e None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:54.369988 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927654be-26a6-457f-b201-72fdf287b45e None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:54.370191 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927654be-26a6-457f-b201-72fdf287b45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:54.387510 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-927654be-26a6-457f-b201-72fdf287b45e None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:54.388680 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-927654be-26a6-457f-b201-72fdf287b45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:54.468132 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 490/957] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:25:54.490437 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7318feaa-0ee1-4437-a813-ff043d7a9294 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:54.506642 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:54.600790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7318feaa-0ee1-4437-a813-ff043d7a9294 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:54.643307 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd0239d-1077-480e-9a13-1780469482d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:54.644115 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0239d-1077-480e-9a13-1780469482d0 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:54.644398 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0239d-1077-480e-9a13-1780469482d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:54.644674 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0239d-1077-480e-9a13-1780469482d0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:54.645335 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd0239d-1077-480e-9a13-1780469482d0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:54.646805 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd0239d-1077-480e-9a13-1780469482d0 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:54.660133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-8bd0239d-1077-480e-9a13-1780469482d0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 19:25:54.660133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:54.660133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 19:25:54.660133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 19:25:54.660133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 19:25:54.660133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 19:25:54.660133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 19:25:54.660133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:25:54.660133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 19:25:54.660133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:54.660133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 19:25:54.660133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:54.660133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:54.660133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:54.660133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:54.660133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:54.660133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:54.661291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:54.661291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:54.661291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:54.661291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:54.661291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:54.661291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:54.661291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 19:25:54.661291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 19:25:54.661291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 19:25:54.661291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 19:25:54.661291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:54.661291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:54.661291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:54.661291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:54.661291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:54.661291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:54.662237 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:54.662237 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:54.662237 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:54.662237 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:54.662237 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:54.662237 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:54.662237 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:54.662237 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:54.662237 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:54.662237 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:54.662237 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 19:25:54.662237 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:25:54.662237 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 19:25:54.662237 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:54.662237 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 468/958] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:25:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:54.668145 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927654be-26a6-457f-b201-72fdf287b45e None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:54.694310 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:54.779230 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927654be-26a6-457f-b201-72fdf287b45e None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:54.815615 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9474e3b5-5f87-48a2-95cd-2c6c0a0229d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:54.816290 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474e3b5-5f87-48a2-95cd-2c6c0a0229d9 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:54.816573 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474e3b5-5f87-48a2-95cd-2c6c0a0229d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:54.816853 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474e3b5-5f87-48a2-95cd-2c6c0a0229d9 None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:54.817566 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9474e3b5-5f87-48a2-95cd-2c6c0a0229d9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:54.817875 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9474e3b5-5f87-48a2-95cd-2c6c0a0229d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:54.819274 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9474e3b5-5f87-48a2-95cd-2c6c0a0229d9 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:54.824652 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 491/959] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Jun 15 19:25:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:54.831148 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd0239d-1077-480e-9a13-1780469482d0 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:54.922035 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:54.950366 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd0239d-1077-480e-9a13-1780469482d0 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:54.988969 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb85e4-33c2-4aaf-8fae-725d30f5d1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:54.989635 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb85e4-33c2-4aaf-8fae-725d30f5d1f7 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:54.989919 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb85e4-33c2-4aaf-8fae-725d30f5d1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:54.990225 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb85e4-33c2-4aaf-8fae-725d30f5d1f7 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:54.990940 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcb85e4-33c2-4aaf-8fae-725d30f5d1f7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:54.991245 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcb85e4-33c2-4aaf-8fae-725d30f5d1f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:54.992763 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcb85e4-33c2-4aaf-8fae-725d30f5d1f7 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:54.997873 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 469/960] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Jun 15 19:25:54 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:55.291253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c259fc-2be6-4696-8ee1-69cb2c601643 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:55.291663 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c259fc-2be6-4696-8ee1-69cb2c601643 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:55.291860 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c259fc-2be6-4696-8ee1-69cb2c601643 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:55.293434 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 492/961] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:55.298802 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734672a-d42c-49f3-85ac-ec32b90d81f1 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:55.299095 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734672a-d42c-49f3-85ac-ec32b90d81f1 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:55.299377 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734672a-d42c-49f3-85ac-ec32b90d81f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:55.316628 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-7734672a-d42c-49f3-85ac-ec32b90d81f1 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:55.317822 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7734672a-d42c-49f3-85ac-ec32b90d81f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:55.394901 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 470/962] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:25:55.423979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c259fc-2be6-4696-8ee1-69cb2c601643 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:55.512506 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:55.530265 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c259fc-2be6-4696-8ee1-69cb2c601643 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:55.566331 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5fe8ab-339f-41e8-a421-f3b4e6e9b2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:55.566924 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5fe8ab-339f-41e8-a421-f3b4e6e9b2a6 None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:55.567210 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5fe8ab-339f-41e8-a421-f3b4e6e9b2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:55.567488 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5fe8ab-339f-41e8-a421-f3b4e6e9b2a6 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:55.568196 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5fe8ab-339f-41e8-a421-f3b4e6e9b2a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:55.569619 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5fe8ab-339f-41e8-a421-f3b4e6e9b2a6 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:55.584783 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.sql.core [None req-0a5fe8ab-339f-41e8-a421-f3b4e6e9b2a6 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 15 19:25:55.584783 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 15 19:25:55.584783 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [parameters: {'user_id': '8a3709564f8a4fffb1b94100e4834c57', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 15 19:25:55.584783 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80223) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 15 19:25:55.593081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-0a5fe8ab-339f-41e8-a421-f3b4e6e9b2a6 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 15 19:25:55.593081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:55.593081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 19:25:55.593081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 19:25:55.593081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 19:25:55.593081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 19:25:55.593081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 19:25:55.593081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 19:25:55.593081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 19:25:55.593081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application conn.query(q) Jun 15 19:25:55.593081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 19:25:55.593081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 19:25:55.593081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 19:25:55.593081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application result.read() Jun 15 19:25:55.593081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 19:25:55.593951 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 19:25:55.593951 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 19:25:55.593951 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 19:25:55.593951 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 19:25:55.593951 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 19:25:55.593951 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 19:25:55.593951 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 19:25:55.593951 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 15 19:25:55.593951 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:55.593951 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 15 19:25:55.593951 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:55.593951 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:55.593951 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 15 19:25:55.593951 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 15 19:25:55.593951 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 15 19:25:55.593951 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 15 19:25:55.593951 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 19:25:55.593951 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application next(self.gen) Jun 15 19:25:55.595187 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 15 19:25:55.595187 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application yield resource Jun 15 19:25:55.595187 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 19:25:55.595187 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application next(self.gen) Jun 15 19:25:55.595187 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 15 19:25:55.595187 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self.session.rollback() Jun 15 19:25:55.595187 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 15 19:25:55.595187 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self.force_reraise() Jun 15 19:25:55.595187 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 15 19:25:55.595187 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise self.value Jun 15 19:25:55.595187 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 15 19:25:55.595187 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 15 19:25:55.595187 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 15 19:25:55.595187 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application session.commit() Jun 15 19:25:55.595187 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 15 19:25:55.595187 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 15 19:25:55.595187 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 15 19:25:55.596284 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._prepare_impl() Jun 15 19:25:55.596284 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 15 19:25:55.596284 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self.session.flush() Jun 15 19:25:55.596284 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 15 19:25:55.596284 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._flush(objects) Jun 15 19:25:55.596284 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 15 19:25:55.596284 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 15 19:25:55.596284 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 15 19:25:55.596284 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application compat.raise_( Jun 15 19:25:55.596284 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 19:25:55.596284 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception Jun 15 19:25:55.596284 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 15 19:25:55.596284 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application flush_context.execute() Jun 15 19:25:55.596284 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 15 19:25:55.596284 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rec.execute(self) Jun 15 19:25:55.596284 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 15 19:25:55.596284 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 15 19:25:55.597272 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 15 19:25:55.597272 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 15 19:25:55.597272 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 15 19:25:55.597272 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 15 19:25:55.597272 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 15 19:25:55.597272 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 15 19:25:55.597272 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 15 19:25:55.597272 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 15 19:25:55.597272 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 15 19:25:55.597272 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 15 19:25:55.597272 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 15 19:25:55.597272 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 15 19:25:55.597272 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 15 19:25:55.597272 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 15 19:25:55.597272 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 19:25:55.597272 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception Jun 15 19:25:55.598654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 19:25:55.598654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 19:25:55.598654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 19:25:55.598654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 19:25:55.598654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 19:25:55.598654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 19:25:55.598654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 19:25:55.598654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application conn.query(q) Jun 15 19:25:55.598654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 19:25:55.598654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 19:25:55.598654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 19:25:55.598654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application result.read() Jun 15 19:25:55.598654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 19:25:55.598654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 19:25:55.598654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 19:25:55.598654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 19:25:55.600253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 19:25:55.600253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 19:25:55.600253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 19:25:55.600253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 19:25:55.600253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 15 19:25:55.600253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 15 19:25:55.600253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [parameters: {'user_id': '8a3709564f8a4fffb1b94100e4834c57', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 15 19:25:55.600253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 15 19:25:55.600253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:55.600253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 19:25:55.600253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:55.600253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:55.600253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:55.600253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:55.600253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:55.601676 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:55.601676 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:55.601676 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:55.601676 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:55.601676 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:55.601676 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:55.601676 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:55.601676 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 15 19:25:55.601676 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 15 19:25:55.601676 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:55.601676 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:55.601676 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:25:55.601676 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:55.601676 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:25:55.601676 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:55.601676 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 15 19:25:55.601676 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 15 19:25:55.603527 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 15 19:25:55.603527 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 15 19:25:55.603527 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 15 19:25:55.603527 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 15 19:25:55.603527 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 15 19:25:55.603527 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:55.603527 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 493/963] 10.222.0.15 () {60 vars in 1131 bytes} [Wed Jun 15 19:25:55 2022] POST /identity/v3/users => generated 160 bytes in 171 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:25:55.610277 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734672a-d42c-49f3-85ac-ec32b90d81f1 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:55.676276 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:55.707779 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734672a-d42c-49f3-85ac-ec32b90d81f1 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:55.743556 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0594bf-546b-473c-825d-742149b42d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:55.744361 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0594bf-546b-473c-825d-742149b42d08 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:55.744700 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0594bf-546b-473c-825d-742149b42d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:55.744982 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0594bf-546b-473c-825d-742149b42d08 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:55.752380 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0594bf-546b-473c-825d-742149b42d08 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:55.753854 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0594bf-546b-473c-825d-742149b42d08 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:55.769507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-dd0594bf-546b-473c-825d-742149b42d08 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 19:25:55.769507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:55.769507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:55.769507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:55.769507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:55.769507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:55.769507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:55.769507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:55.769507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:55.769507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:55.769507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:55.769507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:55.769507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 19:25:55.769507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 19:25:55.769507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 19:25:55.769507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 19:25:55.770547 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:55.770547 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:55.770547 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:25:55.770547 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:55.770547 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:25:55.770547 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:55.770547 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:55.770547 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:55.770547 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:55.770547 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:55.770547 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:55.770547 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:55.770547 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:55.770547 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:55.770547 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:55.770547 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:55.770547 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:55.770547 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:55.771913 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:55.771913 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:55.771913 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 19:25:55.771913 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 19:25:55.771913 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 19:25:55.771913 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 19:25:55.771913 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 19:25:55.771913 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 19:25:55.771913 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 19:25:55.771913 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:55.771913 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 471/964] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:25:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:55.778309 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5fe8ab-339f-41e8-a421-f3b4e6e9b2a6 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:55.886787 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5fe8ab-339f-41e8-a421-f3b4e6e9b2a6 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:55.924085 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e0ca6-9e92-4586-8816-a7823f18b606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:55.924766 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e0ca6-9e92-4586-8816-a7823f18b606 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:55.925051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e0ca6-9e92-4586-8816-a7823f18b606 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:55.925329 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e0ca6-9e92-4586-8816-a7823f18b606 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:55.926173 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e0ca6-9e92-4586-8816-a7823f18b606 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:55.926484 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e0ca6-9e92-4586-8816-a7823f18b606 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:55.928148 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e0ca6-9e92-4586-8816-a7823f18b606 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:55.967436 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 494/965] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Jun 15 19:25:55 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:56.170765 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25e653-c27d-44b6-a9aa-373dcca0c557 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:56.171122 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25e653-c27d-44b6-a9aa-373dcca0c557 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:56.171225 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25e653-c27d-44b6-a9aa-373dcca0c557 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:56.172846 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 472/966] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:56.178583 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc31c25-e01d-49ec-8b5e-0d9a0b583aab None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:56.178881 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc31c25-e01d-49ec-8b5e-0d9a0b583aab None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:56.179162 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc31c25-e01d-49ec-8b5e-0d9a0b583aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:56.186878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:56.199235 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-6dc31c25-e01d-49ec-8b5e-0d9a0b583aab None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:56.200754 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-6dc31c25-e01d-49ec-8b5e-0d9a0b583aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:56.282465 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 495/967] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:25:56.303756 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3e6a7-17bb-48d0-88ab-4101477358f3 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:56.304049 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3e6a7-17bb-48d0-88ab-4101477358f3 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:56.304331 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3e6a7-17bb-48d0-88ab-4101477358f3 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:56.306421 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 473/968] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:56.307930 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc31c25-e01d-49ec-8b5e-0d9a0b583aab None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:56.312313 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdeb7fd-ea94-4da4-926d-114b87c5db7b None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:56.312604 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdeb7fd-ea94-4da4-926d-114b87c5db7b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:56.312881 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdeb7fd-ea94-4da4-926d-114b87c5db7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:56.330378 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-1cdeb7fd-ea94-4da4-926d-114b87c5db7b None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:56.331577 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1cdeb7fd-ea94-4da4-926d-114b87c5db7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:56.409575 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 474/969] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 19:25:56.420230 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc31c25-e01d-49ec-8b5e-0d9a0b583aab None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:56.432328 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdeb7fd-ea94-4da4-926d-114b87c5db7b None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:56.455909 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e8e4d-ae40-4208-bc5e-7ffc3c2b5093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:56.457474 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e8e4d-ae40-4208-bc5e-7ffc3c2b5093 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:56.457944 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e8e4d-ae40-4208-bc5e-7ffc3c2b5093 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:56.458434 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e8e4d-ae40-4208-bc5e-7ffc3c2b5093 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:56.460790 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:56.472691 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e8e4d-ae40-4208-bc5e-7ffc3c2b5093 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:25:56.482599 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-9a5e8e4d-ae40-4208-bc5e-7ffc3c2b5093 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 19:25:56.482599 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:56.482599 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:56.482599 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:56.482599 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:56.482599 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:56.482599 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:56.482599 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:56.482599 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:56.482599 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:56.482599 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:56.482599 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:56.482599 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:25:56.482599 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:25:56.482599 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:25:56.482599 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:25:56.483560 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:25:56.483560 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:25:56.483560 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:25:56.483560 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:25:56.483560 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:56.483560 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:56.483560 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:56.483560 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:56.483560 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:56.483560 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:56.483560 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:56.483560 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:56.483560 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:56.483560 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:56.483560 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:56.483560 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:56.484441 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:56.484441 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:56.484441 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:56.484441 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:56.484441 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:25:56.484441 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:25:56.484441 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:25:56.484441 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:25:56.484441 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:25:56.484441 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:25:56.484441 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 19:25:56.484441 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:56.484441 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 496/970] 10.222.0.15 () {56 vars in 1120 bytes} [Wed Jun 15 19:25:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:56.489571 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e8e4d-ae40-4208-bc5e-7ffc3c2b5093 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:56.501627 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:56.546647 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdeb7fd-ea94-4da4-926d-114b87c5db7b None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:56.584118 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448887d-ba23-405d-8f4a-cb5c4066cb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:56.584931 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448887d-ba23-405d-8f4a-cb5c4066cb1e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:56.585212 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448887d-ba23-405d-8f4a-cb5c4066cb1e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:56.585489 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448887d-ba23-405d-8f4a-cb5c4066cb1e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:56.586169 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2448887d-ba23-405d-8f4a-cb5c4066cb1e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:56.587674 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2448887d-ba23-405d-8f4a-cb5c4066cb1e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:56.593776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e8e4d-ae40-4208-bc5e-7ffc3c2b5093 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:56.600323 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-2448887d-ba23-405d-8f4a-cb5c4066cb1e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 19:25:56.600323 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:56.600323 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 19:25:56.600323 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 19:25:56.600323 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 19:25:56.600323 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 19:25:56.600323 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 19:25:56.600323 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:25:56.600323 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 19:25:56.600323 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:56.600323 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 19:25:56.600323 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:56.600323 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:56.600323 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:56.600323 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:56.600323 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:56.600323 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:56.601537 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:56.601537 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:56.601537 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:56.601537 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:56.601537 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:56.601537 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:56.601537 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 19:25:56.601537 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 19:25:56.601537 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 19:25:56.601537 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 19:25:56.601537 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:56.601537 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:56.601537 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:56.601537 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:56.601537 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:56.601537 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:56.602995 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:56.602995 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:56.602995 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:56.602995 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:56.602995 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:56.602995 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:56.602995 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:56.602995 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:56.602995 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:56.602995 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:56.602995 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 19:25:56.602995 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:25:56.602995 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 19:25:56.602995 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:56.602995 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 475/971] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Jun 15 19:25:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:25:56.607776 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448887d-ba23-405d-8f4a-cb5c4066cb1e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:56.637019 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62867b-ef13-4676-85c9-df101c07fa4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:56.637845 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62867b-ef13-4676-85c9-df101c07fa4d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:56.638177 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62867b-ef13-4676-85c9-df101c07fa4d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:56.638778 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62867b-ef13-4676-85c9-df101c07fa4d None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:56.639821 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c62867b-ef13-4676-85c9-df101c07fa4d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:56.640136 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c62867b-ef13-4676-85c9-df101c07fa4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:56.642165 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c62867b-ef13-4676-85c9-df101c07fa4d None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:56.650849 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 497/972] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Jun 15 19:25:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:56.656328 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62867b-ef13-4676-85c9-df101c07fa4d None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:56.717548 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:56.718686 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448887d-ba23-405d-8f4a-cb5c4066cb1e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:56.746228 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:56.756013 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eafeed-d723-468e-8b00-b4e44e0e320b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:56.756709 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eafeed-d723-468e-8b00-b4e44e0e320b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:56.757000 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eafeed-d723-468e-8b00-b4e44e0e320b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:56.757285 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eafeed-d723-468e-8b00-b4e44e0e320b None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:56.758370 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eafeed-d723-468e-8b00-b4e44e0e320b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:56.758680 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eafeed-d723-468e-8b00-b4e44e0e320b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:56.760067 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eafeed-d723-468e-8b00-b4e44e0e320b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:56.764970 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 476/973] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Jun 15 19:25:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:25:56.770399 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eafeed-d723-468e-8b00-b4e44e0e320b None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:56.771755 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62867b-ef13-4676-85c9-df101c07fa4d None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:56.807934 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3dfaa-2cca-4cb7-b7b2-5afaa6b815e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:56.808728 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3dfaa-2cca-4cb7-b7b2-5afaa6b815e7 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:56.809017 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3dfaa-2cca-4cb7-b7b2-5afaa6b815e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:56.809297 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3dfaa-2cca-4cb7-b7b2-5afaa6b815e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:56.820003 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee3dfaa-2cca-4cb7-b7b2-5afaa6b815e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:56.822052 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee3dfaa-2cca-4cb7-b7b2-5afaa6b815e7 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:56.839186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-3ee3dfaa-2cca-4cb7-b7b2-5afaa6b815e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:56.839186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:56.839186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:56.839186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:56.839186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:56.839186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:56.839186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:56.839186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:56.839186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:56.839186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:56.839186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:56.839186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:56.839186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:56.839186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:56.839186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:56.840127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:56.840127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:56.840127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:56.840127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:56.840127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:56.840127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:56.840127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:56.840127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:56.840127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:56.840127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:56.840127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:56.840127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:56.840127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:56.840127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:56.840127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:56.840127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:56.840127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:56.841079 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:56.841079 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:56.841079 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:56.841079 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:56.841079 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 498/974] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:25:56.845936 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3dfaa-2cca-4cb7-b7b2-5afaa6b815e7 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:56.877018 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eafeed-d723-468e-8b00-b4e44e0e320b None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:56.912837 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea47010-5f12-4ac7-8bf2-1f758c65f605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:56.913745 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea47010-5f12-4ac7-8bf2-1f758c65f605 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:56.914238 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea47010-5f12-4ac7-8bf2-1f758c65f605 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:56.914707 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea47010-5f12-4ac7-8bf2-1f758c65f605 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:56.915565 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea47010-5f12-4ac7-8bf2-1f758c65f605 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:56.916046 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea47010-5f12-4ac7-8bf2-1f758c65f605 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:56.917419 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea47010-5f12-4ac7-8bf2-1f758c65f605 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:56.926962 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 477/975] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Jun 15 19:25:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:25:56.928149 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea47010-5f12-4ac7-8bf2-1f758c65f605 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:56.951252 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3dfaa-2cca-4cb7-b7b2-5afaa6b815e7 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:56.960044 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:56.987521 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352414a0-432c-4734-bf44-842553bdf08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:56.988105 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352414a0-432c-4734-bf44-842553bdf08e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:56.988395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352414a0-432c-4734-bf44-842553bdf08e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:56.988654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352414a0-432c-4734-bf44-842553bdf08e None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:56.989216 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:56.989398 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352414a0-432c-4734-bf44-842553bdf08e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:56.989680 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352414a0-432c-4734-bf44-842553bdf08e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:56.991107 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352414a0-432c-4734-bf44-842553bdf08e None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:57.009312 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 499/976] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:25:57.010692 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352414a0-432c-4734-bf44-842553bdf08e None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:57.043502 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea47010-5f12-4ac7-8bf2-1f758c65f605 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:57.079129 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728abdde-83c9-446d-94a8-24676fdc16e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:57.080107 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728abdde-83c9-446d-94a8-24676fdc16e7 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:57.080573 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728abdde-83c9-446d-94a8-24676fdc16e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:57.081022 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728abdde-83c9-446d-94a8-24676fdc16e7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:57.082019 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728abdde-83c9-446d-94a8-24676fdc16e7 None admin] RBAC: Adding query filter params (interface=admin, service_id=7c4ce0dfa8c0498784b7c3b8d40ddbb3, region_id=RegionOne) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:57.082364 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728abdde-83c9-446d-94a8-24676fdc16e7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:57.083737 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728abdde-83c9-446d-94a8-24676fdc16e7 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:57.089007 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 478/977] 10.222.0.15 () {56 vars in 1249 bytes} [Wed Jun 15 19:25:56 2022] GET /identity/v3/endpoints?service_id=7c4ce0dfa8c0498784b7c3b8d40ddbb3&interface=admin®ion_id=RegionOne => generated 193 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:25:57.094361 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728abdde-83c9-446d-94a8-24676fdc16e7 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:57.122181 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352414a0-432c-4734-bf44-842553bdf08e None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:57.162496 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aaaa51-63bc-4d41-a057-edd2126b9eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:57.163316 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aaaa51-63bc-4d41-a057-edd2126b9eca None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:57.163595 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aaaa51-63bc-4d41-a057-edd2126b9eca None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:57.163869 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aaaa51-63bc-4d41-a057-edd2126b9eca None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:57.171858 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aaaa51-63bc-4d41-a057-edd2126b9eca None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:57.173311 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aaaa51-63bc-4d41-a057-edd2126b9eca None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:57.188198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-07aaaa51-63bc-4d41-a057-edd2126b9eca None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 19:25:57.188198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:57.188198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:57.188198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:57.188198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:57.188198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:57.188198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:57.188198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:57.188198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:57.188198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:57.188198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:57.188198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:57.188198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 19:25:57.188198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 19:25:57.188198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 19:25:57.188198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 19:25:57.189224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:57.189224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:57.189224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:25:57.189224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:57.189224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:25:57.189224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:25:57.189224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:57.189224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:57.189224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:57.189224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:57.189224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:57.189224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:57.189224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:57.189224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:57.189224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:57.189224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:57.189224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:57.189224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:57.190396 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:57.190396 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:57.190396 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 19:25:57.190396 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 19:25:57.190396 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 19:25:57.190396 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 19:25:57.190396 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 19:25:57.190396 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 19:25:57.190396 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 19:25:57.190396 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:57.190396 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 500/978] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Jun 15 19:25:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:57.194909 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aaaa51-63bc-4d41-a057-edd2126b9eca None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:57.208909 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728abdde-83c9-446d-94a8-24676fdc16e7 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:57.223583 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:57.226105 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:57.243035 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6e5f8-9276-4c35-b637-cd2f9f163314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:57.243695 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6e5f8-9276-4c35-b637-cd2f9f163314 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:57.243982 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6e5f8-9276-4c35-b637-cd2f9f163314 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:57.244262 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6e5f8-9276-4c35-b637-cd2f9f163314 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:57.244951 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e6e5f8-9276-4c35-b637-cd2f9f163314 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:57.246367 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e6e5f8-9276-4c35-b637-cd2f9f163314 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:57.252206 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 479/979] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Jun 15 19:25:57 2022] GET /identity/v3/services => generated 1497 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:25:57.253397 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6cbd7-3e2c-4c4f-8a65-e41257c84e00 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:57.253688 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6cbd7-3e2c-4c4f-8a65-e41257c84e00 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:57.253967 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6cbd7-3e2c-4c4f-8a65-e41257c84e00 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:57.255246 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 480/980] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:57.261798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320c15b-2557-407e-9b9b-1af969e848a5 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:57.262108 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320c15b-2557-407e-9b9b-1af969e848a5 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:57.262429 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320c15b-2557-407e-9b9b-1af969e848a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:57.280698 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-5320c15b-2557-407e-9b9b-1af969e848a5 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:57.281954 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-5320c15b-2557-407e-9b9b-1af969e848a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:57.306704 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aaaa51-63bc-4d41-a057-edd2126b9eca None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:57.359347 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f91012-96f6-43c8-b2f9-c5d234224f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:57.360217 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f91012-96f6-43c8-b2f9-c5d234224f65 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:57.360508 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f91012-96f6-43c8-b2f9-c5d234224f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:57.360787 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f91012-96f6-43c8-b2f9-c5d234224f65 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:57.361651 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f91012-96f6-43c8-b2f9-c5d234224f65 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:57.361967 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f91012-96f6-43c8-b2f9-c5d234224f65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:57.363469 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f91012-96f6-43c8-b2f9-c5d234224f65 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:57.373073 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 481/981] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:25:57.389432 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 501/982] 10.222.0.15 () {56 vars in 1149 bytes} [Wed Jun 15 19:25:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:25:57.390724 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320c15b-2557-407e-9b9b-1af969e848a5 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:57.396428 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f91012-96f6-43c8-b2f9-c5d234224f65 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:57.495122 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:57.498628 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320c15b-2557-407e-9b9b-1af969e848a5 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:57.512228 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f91012-96f6-43c8-b2f9-c5d234224f65 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:57.512886 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:57.536947 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7960eb46-65dd-4d8f-aa28-33d02a5ea9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:57.537599 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960eb46-65dd-4d8f-aa28-33d02a5ea9a5 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:57.537881 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960eb46-65dd-4d8f-aa28-33d02a5ea9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:57.538194 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960eb46-65dd-4d8f-aa28-33d02a5ea9a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:57.548631 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921a1561-0545-4e4d-8a60-d8dc32983355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:57.549301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a1561-0545-4e4d-8a60-d8dc32983355 None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:57.549585 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a1561-0545-4e4d-8a60-d8dc32983355 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:57.549864 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a1561-0545-4e4d-8a60-d8dc32983355 None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:57.550641 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921a1561-0545-4e4d-8a60-d8dc32983355 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:57.552034 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921a1561-0545-4e4d-8a60-d8dc32983355 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:57.566533 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 502/983] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Jun 15 19:25:57 2022] POST /identity/v3/services => generated 242 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:25:57.613436 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7960eb46-65dd-4d8f-aa28-33d02a5ea9a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:57.615024 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7960eb46-65dd-4d8f-aa28-33d02a5ea9a5 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:57.726881 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 482/984] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:25:57 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 19:25:57.732645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921a1561-0545-4e4d-8a60-d8dc32983355 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:57.849856 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921a1561-0545-4e4d-8a60-d8dc32983355 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:57.884573 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84690958-896b-449b-965b-1b34e325cc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:57.885382 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84690958-896b-449b-965b-1b34e325cc94 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:57.885662 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84690958-896b-449b-965b-1b34e325cc94 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:57.885933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84690958-896b-449b-965b-1b34e325cc94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:57.895985 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84690958-896b-449b-965b-1b34e325cc94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:57.897419 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84690958-896b-449b-965b-1b34e325cc94 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:57.899196 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:57.919076 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-84690958-896b-449b-965b-1b34e325cc94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:57.919076 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:57.919076 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:57.919076 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:57.919076 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:57.919076 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:57.919076 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:57.919076 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:57.919076 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:57.919076 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:57.919076 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:57.919076 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:57.919076 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:25:57.919076 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:25:57.919076 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:25:57.920049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:25:57.920049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:57.920049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:57.920049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:57.920049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:57.920049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:57.920049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:57.920049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:57.920049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:57.920049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:57.920049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:57.920049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:57.920049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:57.920049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:57.920049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:57.920049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:57.920049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:57.920966 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:25:57.920966 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:25:57.920966 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:25:57.920966 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:57.922177 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 503/985] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:25:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:25:57.926261 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7960eb46-65dd-4d8f-aa28-33d02a5ea9a5 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:57.941683 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:58.029964 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7960eb46-65dd-4d8f-aa28-33d02a5ea9a5 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:58.065187 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0961a1ed-f3c2-4b38-9898-fc1955e634dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:58.065856 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0961a1ed-f3c2-4b38-9898-fc1955e634dd None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:58.066163 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0961a1ed-f3c2-4b38-9898-fc1955e634dd None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:58.066443 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0961a1ed-f3c2-4b38-9898-fc1955e634dd None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:58.067171 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0961a1ed-f3c2-4b38-9898-fc1955e634dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:58.067482 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0961a1ed-f3c2-4b38-9898-fc1955e634dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:58.068851 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0961a1ed-f3c2-4b38-9898-fc1955e634dd None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:58.086012 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 483/986] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:25:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:58.088551 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84690958-896b-449b-965b-1b34e325cc94 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:58.189287 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84690958-896b-449b-965b-1b34e325cc94 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:58.222269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff10f9-d1af-4623-82ea-24fad16b14a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:58.223055 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff10f9-d1af-4623-82ea-24fad16b14a1 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:58.223334 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff10f9-d1af-4623-82ea-24fad16b14a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:58.223607 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff10f9-d1af-4623-82ea-24fad16b14a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:58.230353 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ff10f9-d1af-4623-82ea-24fad16b14a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:58.231793 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ff10f9-d1af-4623-82ea-24fad16b14a1 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:58.245838 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-81ff10f9-d1af-4623-82ea-24fad16b14a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:58.245838 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:58.245838 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:58.245838 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:58.245838 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:58.245838 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:58.245838 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:58.245838 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:58.245838 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:58.245838 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:58.245838 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:58.245838 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:58.245838 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 19:25:58.245838 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 19:25:58.245838 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 19:25:58.246899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 19:25:58.246899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:58.246899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:58.246899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:58.246899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:58.246899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:58.246899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:58.246899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:58.246899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:58.246899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:58.246899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:58.246899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:58.246899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:58.246899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:58.246899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:58.246899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:58.246899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:58.247799 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 19:25:58.247799 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 19:25:58.247799 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:25:58.247799 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:25:58.247799 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:25:58.247799 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:25:58.247799 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:25:58.247799 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:25:58.250165 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 504/987] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:25:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:58.253110 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0961a1ed-f3c2-4b38-9898-fc1955e634dd None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:58.326233 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:58.362748 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0961a1ed-f3c2-4b38-9898-fc1955e634dd None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:58.401652 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad53fca1-bc75-47cb-afd4-fddb03b6f6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:58.402317 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53fca1-bc75-47cb-afd4-fddb03b6f6e8 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:58.402599 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53fca1-bc75-47cb-afd4-fddb03b6f6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:58.402871 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53fca1-bc75-47cb-afd4-fddb03b6f6e8 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:58.403663 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad53fca1-bc75-47cb-afd4-fddb03b6f6e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:58.403971 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad53fca1-bc75-47cb-afd4-fddb03b6f6e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:58.405395 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad53fca1-bc75-47cb-afd4-fddb03b6f6e8 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:58.418443 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 484/988] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jun 15 19:25:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:58.424258 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff10f9-d1af-4623-82ea-24fad16b14a1 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:58.480414 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:58.526925 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff10f9-d1af-4623-82ea-24fad16b14a1 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:58.559755 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b3abf-cf73-40a0-92c3-915aabd442e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:58.560402 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b3abf-cf73-40a0-92c3-915aabd442e4 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:58.560681 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b3abf-cf73-40a0-92c3-915aabd442e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:58.560951 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b3abf-cf73-40a0-92c3-915aabd442e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:58.561776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b3abf-cf73-40a0-92c3-915aabd442e4 None admin] RBAC: Adding query filter params (role.id=34c882c657e2470e9bbe75d07694392d, scope.project.id=7fb8cc0eb560465e972ffd45867ee73b, user.id=67aa048d409c46e9a729aec9d1b2286e) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:58.562150 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b3abf-cf73-40a0-92c3-915aabd442e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:58.563642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b3abf-cf73-40a0-92c3-915aabd442e4 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:58.569624 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 505/989] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:25:58 2022] GET /identity/v3/role_assignments?user.id=67aa048d409c46e9a729aec9d1b2286e&scope.project.id=7fb8cc0eb560465e972ffd45867ee73b&role.id=34c882c657e2470e9bbe75d07694392d => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:59.377586 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696324e2-9b98-4240-bf68-f3b2c1db86b7 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:59.378136 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696324e2-9b98-4240-bf68-f3b2c1db86b7 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:59.378136 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696324e2-9b98-4240-bf68-f3b2c1db86b7 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:59.379662 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 485/990] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:25:59.385474 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5e8bf-41d7-4e7d-b21e-f513d9014b99 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:59.385771 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5e8bf-41d7-4e7d-b21e-f513d9014b99 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:59.386048 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5e8bf-41d7-4e7d-b21e-f513d9014b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:59.403279 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-c0b5e8bf-41d7-4e7d-b21e-f513d9014b99 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:59.404458 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-c0b5e8bf-41d7-4e7d-b21e-f513d9014b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:59.469713 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 506/991] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:59 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:25:59.492929 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696324e2-9b98-4240-bf68-f3b2c1db86b7 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:59.597623 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696324e2-9b98-4240-bf68-f3b2c1db86b7 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:59.628480 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:59.635201 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f553fa-d1f4-41e6-96a4-d34fd4abac21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:59.636202 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f553fa-d1f4-41e6-96a4-d34fd4abac21 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:59.636631 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f553fa-d1f4-41e6-96a4-d34fd4abac21 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:59.637058 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f553fa-d1f4-41e6-96a4-d34fd4abac21 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:59.637852 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f553fa-d1f4-41e6-96a4-d34fd4abac21 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:59.639381 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f553fa-d1f4-41e6-96a4-d34fd4abac21 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:59.651672 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-99f553fa-d1f4-41e6-96a4-d34fd4abac21 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 19:25:59.651672 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:59.651672 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 19:25:59.651672 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 19:25:59.651672 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 19:25:59.651672 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 19:25:59.651672 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 19:25:59.651672 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:25:59.651672 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 19:25:59.651672 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:59.651672 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 19:25:59.651672 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:59.651672 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:25:59.651672 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:25:59.651672 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:25:59.651672 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:25:59.651672 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:25:59.652765 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:25:59.652765 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:25:59.652765 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:25:59.652765 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:25:59.652765 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:25:59.652765 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:25:59.652765 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 19:25:59.652765 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 19:25:59.652765 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 19:25:59.652765 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 19:25:59.652765 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:25:59.652765 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:25:59.652765 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:25:59.652765 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:25:59.652765 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:25:59.652765 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:25:59.653617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:25:59.653617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:25:59.653617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:25:59.653617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:25:59.653617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:25:59.653617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:25:59.653617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:25:59.653617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:25:59.653617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:25:59.653617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:25:59.653617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 19:25:59.653617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:25:59.653617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 19:25:59.653617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:25:59.658305 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 486/992] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Jun 15 19:25:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:25:59.659991 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b5e8bf-41d7-4e7d-b21e-f513d9014b99 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:59.715798 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:59.732581 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc9e45-065d-40b0-920c-8706b67a8ec0 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:59.732866 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc9e45-065d-40b0-920c-8706b67a8ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:59.733139 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc9e45-065d-40b0-920c-8706b67a8ec0 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:59.734683 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 487/993] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 19:25:59.739267 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d01ed69-49b0-41b7-9b49-2abd1a5ab120 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:59.739548 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d01ed69-49b0-41b7-9b49-2abd1a5ab120 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:59.739828 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d01ed69-49b0-41b7-9b49-2abd1a5ab120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:59.757189 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-6d01ed69-49b0-41b7-9b49-2abd1a5ab120 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:25:59.758653 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6d01ed69-49b0-41b7-9b49-2abd1a5ab120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:25:59.772458 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b5e8bf-41d7-4e7d-b21e-f513d9014b99 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:59.818538 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb01a1-4d0e-49e7-a30b-18add5200f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:59.819499 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb01a1-4d0e-49e7-a30b-18add5200f76 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:59.819782 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb01a1-4d0e-49e7-a30b-18add5200f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:59.820453 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb01a1-4d0e-49e7-a30b-18add5200f76 None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:59.821468 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb01a1-4d0e-49e7-a30b-18add5200f76 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:59.821773 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb01a1-4d0e-49e7-a30b-18add5200f76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:59.823957 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb01a1-4d0e-49e7-a30b-18add5200f76 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:59.827526 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 488/994] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:25:59 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 19:25:59.830234 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 507/995] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Jun 15 19:25:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:59.835447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d01ed69-49b0-41b7-9b49-2abd1a5ab120 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:59.850111 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb01a1-4d0e-49e7-a30b-18add5200f76 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:25:59.941457 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d01ed69-49b0-41b7-9b49-2abd1a5ab120 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:59.956793 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb01a1-4d0e-49e7-a30b-18add5200f76 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:25:59.970404 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:59.971942 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:25:59.980295 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf38e3-ff8f-4c3c-9780-fb7994076758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:25:59.980962 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf38e3-ff8f-4c3c-9780-fb7994076758 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:25:59.981246 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf38e3-ff8f-4c3c-9780-fb7994076758 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:25:59.981521 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf38e3-ff8f-4c3c-9780-fb7994076758 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:25:59.982327 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cf38e3-ff8f-4c3c-9780-fb7994076758 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:25:59.982640 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cf38e3-ff8f-4c3c-9780-fb7994076758 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:25:59.984004 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cf38e3-ff8f-4c3c-9780-fb7994076758 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:25:59.993425 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 489/996] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Jun 15 19:25:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:25:59.994357 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf38e3-ff8f-4c3c-9780-fb7994076758 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:00.001325 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ece6c-60d9-462f-851d-22f1497590bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:00.002126 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ece6c-60d9-462f-851d-22f1497590bc None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:00.002410 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ece6c-60d9-462f-851d-22f1497590bc None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:00.002682 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ece6c-60d9-462f-851d-22f1497590bc None admin] PATH_INFO: `/v3/services/compute` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:00.003341 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ece6c-60d9-462f-851d-22f1497590bc None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:00.004669 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ece6c-60d9-462f-851d-22f1497590bc None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:00.016587 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-ea7ece6c-60d9-462f-851d-22f1497590bc None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 19:26:00.016587 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:26:00.016587 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 19:26:00.016587 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 19:26:00.016587 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 19:26:00.016587 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 19:26:00.016587 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 19:26:00.016587 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:26:00.016587 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 19:26:00.016587 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:26:00.016587 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 19:26:00.016587 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:26:00.016587 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:26:00.016587 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:26:00.016587 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:26:00.016587 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:26:00.016587 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:26:00.017571 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:26:00.017571 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:26:00.017571 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:26:00.017571 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:26:00.017571 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:26:00.017571 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:26:00.017571 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 19:26:00.017571 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 19:26:00.017571 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 19:26:00.017571 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 19:26:00.017571 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:26:00.017571 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:26:00.017571 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:26:00.017571 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:26:00.017571 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:26:00.017571 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:26:00.018549 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:26:00.018549 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:26:00.018549 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:26:00.018549 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:26:00.018549 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:26:00.018549 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:26:00.018549 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:26:00.018549 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:26:00.018549 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:26:00.018549 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:26:00.018549 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 19:26:00.018549 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:26:00.018549 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 19:26:00.018549 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:26:00.018549 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 508/997] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:25:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:26:00.023312 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ece6c-60d9-462f-851d-22f1497590bc None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:00.099392 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf38e3-ff8f-4c3c-9780-fb7994076758 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:00.128700 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ece6c-60d9-462f-851d-22f1497590bc None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:00.134707 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7366f0ee-3530-4ea9-9c3b-f0578696a0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:00.135369 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7366f0ee-3530-4ea9-9c3b-f0578696a0de None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:00.135654 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7366f0ee-3530-4ea9-9c3b-f0578696a0de None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:00.135929 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7366f0ee-3530-4ea9-9c3b-f0578696a0de None admin] PATH_INFO: `/v3/endpoints` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:00.136669 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7366f0ee-3530-4ea9-9c3b-f0578696a0de None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:00.138033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7366f0ee-3530-4ea9-9c3b-f0578696a0de None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:00.162132 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 490/998] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Jun 15 19:25:59 2022] POST /identity/v3/endpoints => generated 325 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:26:00.184400 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55f3ad-be0f-43e6-a43a-d5dcdec2fb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:00.185077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55f3ad-be0f-43e6-a43a-d5dcdec2fb6e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:00.185363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55f3ad-be0f-43e6-a43a-d5dcdec2fb6e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:00.185635 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55f3ad-be0f-43e6-a43a-d5dcdec2fb6e None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:00.186379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c55f3ad-be0f-43e6-a43a-d5dcdec2fb6e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:00.186682 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c55f3ad-be0f-43e6-a43a-d5dcdec2fb6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:00.188060 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c55f3ad-be0f-43e6-a43a-d5dcdec2fb6e None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:00.193720 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 509/999] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Jun 15 19:26:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:26:00.199826 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7366f0ee-3530-4ea9-9c3b-f0578696a0de None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:00.256246 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:00.315119 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7366f0ee-3530-4ea9-9c3b-f0578696a0de None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:00.352789 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d71bb8-b202-4afc-a67d-e524da312a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:00.353683 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d71bb8-b202-4afc-a67d-e524da312a3b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:00.353968 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d71bb8-b202-4afc-a67d-e524da312a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:00.354277 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d71bb8-b202-4afc-a67d-e524da312a3b None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:00.355000 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d71bb8-b202-4afc-a67d-e524da312a3b None admin] RBAC: Adding query filter params (type=compute) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:00.355310 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d71bb8-b202-4afc-a67d-e524da312a3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:00.356690 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d71bb8-b202-4afc-a67d-e524da312a3b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:00.362011 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 491/1000] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Jun 15 19:26:00 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:00.368139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55f3ad-be0f-43e6-a43a-d5dcdec2fb6e None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:00.405060 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:00.470378 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55f3ad-be0f-43e6-a43a-d5dcdec2fb6e None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:00.503978 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6dfe77-298f-4ec9-9e8c-d0504f0cfb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:00.504635 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6dfe77-298f-4ec9-9e8c-d0504f0cfb8c None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:00.504919 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6dfe77-298f-4ec9-9e8c-d0504f0cfb8c None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:00.505194 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6dfe77-298f-4ec9-9e8c-d0504f0cfb8c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:00.505978 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6dfe77-298f-4ec9-9e8c-d0504f0cfb8c None admin] RBAC: Adding query filter params (interface=public, service_id=1d995c4529f14b4bba15924e33c1a56e, region_id=RegionOne) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:00.506330 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6dfe77-298f-4ec9-9e8c-d0504f0cfb8c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:00.507938 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6dfe77-298f-4ec9-9e8c-d0504f0cfb8c None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:00.513247 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 510/1001] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Jun 15 19:26:00 2022] GET /identity/v3/endpoints?service_id=1d995c4529f14b4bba15924e33c1a56e&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:00.518787 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d71bb8-b202-4afc-a67d-e524da312a3b None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:00.621070 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d71bb8-b202-4afc-a67d-e524da312a3b None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:00.635065 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:00.657872 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb2dfd1-72a7-44d6-9091-ae4ff48fced9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:00.658510 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2dfd1-72a7-44d6-9091-ae4ff48fced9 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:00.658793 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2dfd1-72a7-44d6-9091-ae4ff48fced9 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:00.659070 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2dfd1-72a7-44d6-9091-ae4ff48fced9 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:00.659776 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb2dfd1-72a7-44d6-9091-ae4ff48fced9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:00.661199 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb2dfd1-72a7-44d6-9091-ae4ff48fced9 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:00.674155 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 492/1002] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Jun 15 19:26:00 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:26:00.727991 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488871a-269a-4158-9d08-9ee7d65b9276 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:00.728556 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488871a-269a-4158-9d08-9ee7d65b9276 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:00.728556 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488871a-269a-4158-9d08-9ee7d65b9276 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:00.729953 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 511/1003] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:26:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:26:00.735670 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908b91e-8292-4e19-871f-5e2418cb7fda None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:00.735966 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908b91e-8292-4e19-871f-5e2418cb7fda None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:00.736247 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908b91e-8292-4e19-871f-5e2418cb7fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:00.753755 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-a908b91e-8292-4e19-871f-5e2418cb7fda None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:26:00.755013 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a908b91e-8292-4e19-871f-5e2418cb7fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:26:00.830944 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 493/1004] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:26:00 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:26:00.853897 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b488871a-269a-4158-9d08-9ee7d65b9276 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:00.956386 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:00.960677 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b488871a-269a-4158-9d08-9ee7d65b9276 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:00.997231 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f140d6-839a-4618-9a19-7fe966bd2168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:00.998021 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f140d6-839a-4618-9a19-7fe966bd2168 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:00.998350 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f140d6-839a-4618-9a19-7fe966bd2168 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:00.998631 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f140d6-839a-4618-9a19-7fe966bd2168 None admin] PATH_INFO: `/v3/services/image` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:00.999286 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f140d6-839a-4618-9a19-7fe966bd2168 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:01.000756 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f140d6-839a-4618-9a19-7fe966bd2168 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:01.012932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-f3f140d6-839a-4618-9a19-7fe966bd2168 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 19:26:01.012932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:26:01.012932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 19:26:01.012932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 19:26:01.012932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 19:26:01.012932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 19:26:01.012932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 19:26:01.012932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:26:01.012932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 19:26:01.012932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:26:01.012932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 19:26:01.012932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:26:01.012932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:26:01.012932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:26:01.012932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:26:01.012932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:26:01.012932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:26:01.013922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:26:01.013922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:26:01.013922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:26:01.013922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:26:01.013922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:26:01.013922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:26:01.013922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 19:26:01.013922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 19:26:01.013922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 19:26:01.013922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 19:26:01.013922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:26:01.013922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:26:01.013922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:26:01.013922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:26:01.013922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:26:01.013922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:26:01.014812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:26:01.014812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:26:01.014812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:26:01.014812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:26:01.014812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:26:01.014812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:26:01.014812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:26:01.014812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:26:01.014812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:26:01.014812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:26:01.014812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 19:26:01.014812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:26:01.014812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 19:26:01.014812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:26:01.014812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 512/1005] 10.222.0.15 () {56 vars in 1105 bytes} [Wed Jun 15 19:26:00 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:26:01.021504 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908b91e-8292-4e19-871f-5e2418cb7fda None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:01.123193 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908b91e-8292-4e19-871f-5e2418cb7fda None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:01.132423 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:01.158362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da400fd5-e3a3-44ea-8647-f94f5bbadcd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:01.159035 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da400fd5-e3a3-44ea-8647-f94f5bbadcd4 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:01.159319 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da400fd5-e3a3-44ea-8647-f94f5bbadcd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:01.159594 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da400fd5-e3a3-44ea-8647-f94f5bbadcd4 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:01.160301 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da400fd5-e3a3-44ea-8647-f94f5bbadcd4 None admin] RBAC: Adding query filter params (name=image) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:01.160609 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da400fd5-e3a3-44ea-8647-f94f5bbadcd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:01.161975 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da400fd5-e3a3-44ea-8647-f94f5bbadcd4 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:01.169308 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 494/1006] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:26:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:01.175055 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f140d6-839a-4618-9a19-7fe966bd2168 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:01.279845 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f140d6-839a-4618-9a19-7fe966bd2168 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:01.314126 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53da0355-fd3a-403b-a1a7-cb4e5712553d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:01.314126 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da0355-fd3a-403b-a1a7-cb4e5712553d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:01.314126 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da0355-fd3a-403b-a1a7-cb4e5712553d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:01.314437 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da0355-fd3a-403b-a1a7-cb4e5712553d None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:01.315084 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53da0355-fd3a-403b-a1a7-cb4e5712553d None admin] RBAC: Adding query filter params (type=image) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:01.315387 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53da0355-fd3a-403b-a1a7-cb4e5712553d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:01.316743 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53da0355-fd3a-403b-a1a7-cb4e5712553d None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:01.330212 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 513/1007] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:26:01 2022] GET /identity/v3/services?type=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:02.314304 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82131394-452b-4dfc-ac6b-4af88192f2ec None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:02.314847 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82131394-452b-4dfc-ac6b-4af88192f2ec None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:02.314847 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82131394-452b-4dfc-ac6b-4af88192f2ec None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:02.316253 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 495/1008] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:26:02.321372 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb87fc6-23f8-4826-aff2-7ee249ecda15 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:02.321778 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb87fc6-23f8-4826-aff2-7ee249ecda15 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:02.321985 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb87fc6-23f8-4826-aff2-7ee249ecda15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:02.339505 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-ffb87fc6-23f8-4826-aff2-7ee249ecda15 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:26:02.340706 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-ffb87fc6-23f8-4826-aff2-7ee249ecda15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:26:02.365239 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:02.413367 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 514/1009] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:26:02 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:26:02.437236 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82131394-452b-4dfc-ac6b-4af88192f2ec None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:02.539676 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82131394-452b-4dfc-ac6b-4af88192f2ec None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:02.573435 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d961120-36de-4edf-965a-988600794937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:02.574191 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d961120-36de-4edf-965a-988600794937 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:02.574475 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d961120-36de-4edf-965a-988600794937 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:02.574747 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d961120-36de-4edf-965a-988600794937 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:02.575927 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d961120-36de-4edf-965a-988600794937 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:02.577368 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d961120-36de-4edf-965a-988600794937 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:02.579162 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 496/1010] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:26:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:02.584648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb87fc6-23f8-4826-aff2-7ee249ecda15 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:02.682509 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb87fc6-23f8-4826-aff2-7ee249ecda15 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:02.715917 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d057f-0aba-4f46-90b6-c39cfeeab697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:02.716576 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d057f-0aba-4f46-90b6-c39cfeeab697 None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:02.716863 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d057f-0aba-4f46-90b6-c39cfeeab697 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:02.717144 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d057f-0aba-4f46-90b6-c39cfeeab697 None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:02.717861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12d057f-0aba-4f46-90b6-c39cfeeab697 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:02.719313 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12d057f-0aba-4f46-90b6-c39cfeeab697 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:02.749119 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 515/1011] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Jun 15 19:26:02 2022] POST /identity/v3/projects => generated 311 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:26:02.822400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465819d-67ea-49c0-9531-6e3f4f2c2552 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:02.822727 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465819d-67ea-49c0-9531-6e3f4f2c2552 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:02.822955 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465819d-67ea-49c0-9531-6e3f4f2c2552 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:02.824576 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 497/1012] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:26:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:26:02.830122 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fe7ab-d47a-4ec6-91fc-fc896ea707d9 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:02.830401 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fe7ab-d47a-4ec6-91fc-fc896ea707d9 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:02.830682 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fe7ab-d47a-4ec6-91fc-fc896ea707d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:02.848069 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-f00fe7ab-d47a-4ec6-91fc-fc896ea707d9 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:26:02.849279 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-f00fe7ab-d47a-4ec6-91fc-fc896ea707d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:26:02.852525 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:02.928288 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 516/1013] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:26:02 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:26:02.951556 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1465819d-67ea-49c0-9531-6e3f4f2c2552 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:02.998926 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:03.062046 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1465819d-67ea-49c0-9531-6e3f4f2c2552 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:03.097744 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4caef62-e060-4ffa-842b-ed2c2b9b8127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:03.098490 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4caef62-e060-4ffa-842b-ed2c2b9b8127 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:03.098776 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4caef62-e060-4ffa-842b-ed2c2b9b8127 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:03.099049 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4caef62-e060-4ffa-842b-ed2c2b9b8127 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:03.099700 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4caef62-e060-4ffa-842b-ed2c2b9b8127 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:03.101061 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4caef62-e060-4ffa-842b-ed2c2b9b8127 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:03.113562 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-d4caef62-e060-4ffa-842b-ed2c2b9b8127 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 19:26:03.113562 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:26:03.113562 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 19:26:03.113562 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 19:26:03.113562 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 19:26:03.113562 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 19:26:03.113562 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 19:26:03.113562 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:26:03.113562 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 19:26:03.113562 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:26:03.113562 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 19:26:03.113562 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:26:03.113562 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:26:03.113562 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:26:03.113562 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:26:03.113562 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:26:03.113562 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:26:03.114783 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:26:03.114783 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:26:03.114783 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:26:03.114783 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:26:03.114783 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:26:03.114783 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:26:03.114783 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 19:26:03.114783 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 19:26:03.114783 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 19:26:03.114783 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 19:26:03.114783 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:26:03.114783 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:26:03.114783 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:26:03.114783 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:26:03.114783 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:26:03.114783 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:26:03.115622 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:26:03.115622 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:26:03.115622 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:26:03.115622 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:26:03.115622 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:26:03.115622 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:26:03.115622 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:26:03.115622 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:26:03.115622 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:26:03.115622 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:26:03.115622 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 19:26:03.115622 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:26:03.115622 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 19:26:03.115622 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:26:03.115622 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 498/1014] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:26:02 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:26:03.120834 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00fe7ab-d47a-4ec6-91fc-fc896ea707d9 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:03.221293 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00fe7ab-d47a-4ec6-91fc-fc896ea707d9 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:03.254979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4d768-a1c2-493a-8e16-fe1462b8bcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:03.255657 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4d768-a1c2-493a-8e16-fe1462b8bcd8 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:03.255939 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4d768-a1c2-493a-8e16-fe1462b8bcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:03.256214 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4d768-a1c2-493a-8e16-fe1462b8bcd8 None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:03.256920 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f4d768-a1c2-493a-8e16-fe1462b8bcd8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:03.257227 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f4d768-a1c2-493a-8e16-fe1462b8bcd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:03.259417 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f4d768-a1c2-493a-8e16-fe1462b8bcd8 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:03.268873 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 517/1015] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Jun 15 19:26:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:26:03.270457 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4caef62-e060-4ffa-842b-ed2c2b9b8127 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:03.387478 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4caef62-e060-4ffa-842b-ed2c2b9b8127 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:03.390605 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:03.415787 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9e96e-d7d3-405a-90c7-902696c0310c None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:03.416274 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9e96e-d7d3-405a-90c7-902696c0310c None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:03.416363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9e96e-d7d3-405a-90c7-902696c0310c None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:03.417808 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 518/1016] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:26:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 19:26:03.422308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0464ae-0707-471f-bb65-9cf0896e5a3f None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:03.422580 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0464ae-0707-471f-bb65-9cf0896e5a3f None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:03.422858 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0464ae-0707-471f-bb65-9cf0896e5a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:03.424763 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0eb06d-ebb3-4da1-9786-f910b1e33a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:03.425436 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0eb06d-ebb3-4da1-9786-f910b1e33a41 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:03.425724 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0eb06d-ebb3-4da1-9786-f910b1e33a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:03.426003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0eb06d-ebb3-4da1-9786-f910b1e33a41 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:03.426744 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0eb06d-ebb3-4da1-9786-f910b1e33a41 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:03.427054 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0eb06d-ebb3-4da1-9786-f910b1e33a41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:03.428432 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0eb06d-ebb3-4da1-9786-f910b1e33a41 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:03.430805 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:03.434039 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 499/1017] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Jun 15 19:26:03 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:03.439598 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0eb06d-ebb3-4da1-9786-f910b1e33a41 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:03.441650 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-da0464ae-0707-471f-bb65-9cf0896e5a3f None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:26:03.442924 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-da0464ae-0707-471f-bb65-9cf0896e5a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:26:03.514950 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 519/1018] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:26:03 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 19:26:03.537748 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0464ae-0707-471f-bb65-9cf0896e5a3f None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:03.548317 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0eb06d-ebb3-4da1-9786-f910b1e33a41 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:03.589920 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d966e6-a9a4-4213-9a7a-118bcabce7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:03.590601 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d966e6-a9a4-4213-9a7a-118bcabce7be None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:03.590882 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d966e6-a9a4-4213-9a7a-118bcabce7be None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:03.591576 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d966e6-a9a4-4213-9a7a-118bcabce7be None admin] PATH_INFO: `/v3/endpoints` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:03.592374 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d966e6-a9a4-4213-9a7a-118bcabce7be None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:03.593739 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d966e6-a9a4-4213-9a7a-118bcabce7be None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:03.615484 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 500/1019] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Jun 15 19:26:03 2022] POST /identity/v3/endpoints => generated 334 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:26:03.644699 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0464ae-0707-471f-bb65-9cf0896e5a3f None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:03.685266 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:03.692176 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e57000-9002-4b64-bb97-a5f16ce94aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:03.693399 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e57000-9002-4b64-bb97-a5f16ce94aff None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:03.693678 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e57000-9002-4b64-bb97-a5f16ce94aff None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:03.693952 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e57000-9002-4b64-bb97-a5f16ce94aff None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:03.694735 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e57000-9002-4b64-bb97-a5f16ce94aff None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:03.696178 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e57000-9002-4b64-bb97-a5f16ce94aff None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:03.713343 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 520/1020] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Jun 15 19:26:03 2022] POST /identity/v3/services => generated 242 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:26:04.966245 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec58ea6-8b0a-4d6e-b0d8-166da714cefe None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:04.966772 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec58ea6-8b0a-4d6e-b0d8-166da714cefe None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:04.966851 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec58ea6-8b0a-4d6e-b0d8-166da714cefe None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:04.968208 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 501/1021] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:26:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:26:04.973198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd70fe1-41b7-4b2f-abc8-3a1bc16456fb None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:04.973591 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd70fe1-41b7-4b2f-abc8-3a1bc16456fb None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:04.973785 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd70fe1-41b7-4b2f-abc8-3a1bc16456fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:04.990157 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-bdd70fe1-41b7-4b2f-abc8-3a1bc16456fb None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:26:04.991320 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-bdd70fe1-41b7-4b2f-abc8-3a1bc16456fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:26:05.063591 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 521/1022] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:26:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:26:05.088415 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec58ea6-8b0a-4d6e-b0d8-166da714cefe None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:05.114365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:05.194327 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec58ea6-8b0a-4d6e-b0d8-166da714cefe None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:05.228477 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f311bd3d-61b9-4ef6-951a-db069d6ad1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:05.229203 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311bd3d-61b9-4ef6-951a-db069d6ad1c4 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:05.229486 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311bd3d-61b9-4ef6-951a-db069d6ad1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:05.229762 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311bd3d-61b9-4ef6-951a-db069d6ad1c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:05.230977 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f311bd3d-61b9-4ef6-951a-db069d6ad1c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:05.232425 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f311bd3d-61b9-4ef6-951a-db069d6ad1c4 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:05.234285 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 502/1023] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:26:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:05.239629 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd70fe1-41b7-4b2f-abc8-3a1bc16456fb None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:05.351048 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd70fe1-41b7-4b2f-abc8-3a1bc16456fb None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:05.368740 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:05.388626 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da97b196-4005-45cb-a5a0-97cf9ca08647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:05.389294 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97b196-4005-45cb-a5a0-97cf9ca08647 None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:05.389572 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97b196-4005-45cb-a5a0-97cf9ca08647 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:05.389842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97b196-4005-45cb-a5a0-97cf9ca08647 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:05.390582 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da97b196-4005-45cb-a5a0-97cf9ca08647 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:05.392058 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da97b196-4005-45cb-a5a0-97cf9ca08647 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:05.412875 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 522/1024] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Jun 15 19:26:05 2022] POST /identity/v3/users => generated 285 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:26:05.847747 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed128fc-c656-4ee2-be2b-7331af99c9d1 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:05.848036 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed128fc-c656-4ee2-be2b-7331af99c9d1 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:05.848319 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed128fc-c656-4ee2-be2b-7331af99c9d1 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:05.849731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 503/1025] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:26:05.856332 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e7474-dfae-429b-8f3c-ed216c54a2bd None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:05.856628 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e7474-dfae-429b-8f3c-ed216c54a2bd None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:05.856908 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e7474-dfae-429b-8f3c-ed216c54a2bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:05.875636 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-cc4e7474-dfae-429b-8f3c-ed216c54a2bd None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:26:05.877208 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-cc4e7474-dfae-429b-8f3c-ed216c54a2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:26:05.891957 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3abf5-1cb7-4a68-a348-2763da05187e None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:05.892522 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3abf5-1cb7-4a68-a348-2763da05187e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:05.892868 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3abf5-1cb7-4a68-a348-2763da05187e None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:05.894499 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 504/1026] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 19:26:05.900884 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3a87a-963a-4bdb-9206-654259c3d780 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:05.901174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3a87a-963a-4bdb-9206-654259c3d780 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:05.901451 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3a87a-963a-4bdb-9206-654259c3d780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:05.919064 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-5ee3a87a-963a-4bdb-9206-654259c3d780 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:26:05.920231 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-5ee3a87a-963a-4bdb-9206-654259c3d780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:26:05.950738 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 523/1027] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:26:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:26:05.974335 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e7474-dfae-429b-8f3c-ed216c54a2bd None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:05.991282 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 505/1028] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:26:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 19:26:06.013096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3a87a-963a-4bdb-9206-654259c3d780 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:06.037904 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:06.089278 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e7474-dfae-429b-8f3c-ed216c54a2bd None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:06.090963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:06.116207 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3a87a-963a-4bdb-9206-654259c3d780 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:06.129386 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b998b50-a8b3-4072-a5ce-7d0e188acd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:06.130239 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b998b50-a8b3-4072-a5ce-7d0e188acd95 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:06.130528 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b998b50-a8b3-4072-a5ce-7d0e188acd95 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:06.130822 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b998b50-a8b3-4072-a5ce-7d0e188acd95 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:06.142341 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b998b50-a8b3-4072-a5ce-7d0e188acd95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:26:06.150740 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-5b998b50-a8b3-4072-a5ce-7d0e188acd95 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 19:26:06.150740 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:26:06.150740 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:26:06.150740 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:26:06.150740 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:26:06.150740 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:26:06.150740 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:26:06.150740 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:26:06.150740 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:26:06.150740 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:26:06.150740 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:26:06.150740 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:26:06.150740 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:26:06.150740 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:26:06.150740 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:26:06.150740 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:26:06.151882 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:26:06.151882 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:26:06.151882 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:26:06.151882 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:26:06.151882 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:26:06.151882 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:26:06.151882 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:26:06.151882 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:26:06.151882 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:26:06.151882 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:26:06.151882 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:26:06.151882 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:26:06.151882 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:26:06.151882 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:26:06.151882 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:26:06.151882 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:26:06.152757 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:26:06.152757 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:26:06.152757 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:26:06.152757 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:26:06.152757 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:26:06.152757 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:26:06.152757 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:26:06.152757 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:26:06.152757 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:26:06.152757 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:26:06.152757 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 19:26:06.152757 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:26:06.152757 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 524/1029] 10.222.0.15 () {56 vars in 1120 bytes} [Wed Jun 15 19:26:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:26:06.155824 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd3f29-45ec-466d-83a3-a3904e406b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:06.156842 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd3f29-45ec-466d-83a3-a3904e406b5d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:06.156948 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd3f29-45ec-466d-83a3-a3904e406b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:06.157153 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd3f29-45ec-466d-83a3-a3904e406b5d None admin] PATH_INFO: `/v3/services/image` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:06.157817 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcd3f29-45ec-466d-83a3-a3904e406b5d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:06.158020 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b998b50-a8b3-4072-a5ce-7d0e188acd95 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:06.159259 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcd3f29-45ec-466d-83a3-a3904e406b5d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:06.172468 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-edcd3f29-45ec-466d-83a3-a3904e406b5d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 19:26:06.172468 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:26:06.172468 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 19:26:06.172468 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 19:26:06.172468 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 19:26:06.172468 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 19:26:06.172468 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 19:26:06.172468 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:26:06.172468 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 19:26:06.172468 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:26:06.172468 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 19:26:06.172468 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:26:06.172468 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:26:06.172468 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:26:06.172468 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:26:06.172468 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:26:06.172468 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:26:06.173511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:26:06.173511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:26:06.173511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:26:06.173511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:26:06.173511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:26:06.173511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:26:06.173511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 19:26:06.173511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 19:26:06.173511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 19:26:06.173511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 19:26:06.173511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:26:06.173511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:26:06.173511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:26:06.173511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:26:06.173511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:26:06.173511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:26:06.174403 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:26:06.174403 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:26:06.174403 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:26:06.174403 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:26:06.174403 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:26:06.174403 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:26:06.174403 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:26:06.174403 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:26:06.174403 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:26:06.174403 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:26:06.174403 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 19:26:06.174403 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:26:06.174403 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 19:26:06.174403 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:26:06.174403 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 506/1030] 10.222.0.15 () {56 vars in 1105 bytes} [Wed Jun 15 19:26:06 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:26:06.180212 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd3f29-45ec-466d-83a3-a3904e406b5d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:06.265846 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b998b50-a8b3-4072-a5ce-7d0e188acd95 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:06.271765 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:06.288118 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd3f29-45ec-466d-83a3-a3904e406b5d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:06.304191 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e17a87-d9b2-4ce8-b400-5161e3f1dca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:06.305398 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e17a87-d9b2-4ce8-b400-5161e3f1dca3 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:06.305682 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e17a87-d9b2-4ce8-b400-5161e3f1dca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:06.305958 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e17a87-d9b2-4ce8-b400-5161e3f1dca3 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:06.306735 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e17a87-d9b2-4ce8-b400-5161e3f1dca3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:06.307044 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e17a87-d9b2-4ce8-b400-5161e3f1dca3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:06.308406 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e17a87-d9b2-4ce8-b400-5161e3f1dca3 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:06.317032 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 525/1031] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Jun 15 19:26:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:06.322360 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e17a87-d9b2-4ce8-b400-5161e3f1dca3 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:06.324459 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d9655-005c-417e-a423-88bdfdd1fa7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:06.325124 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d9655-005c-417e-a423-88bdfdd1fa7a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:06.325402 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d9655-005c-417e-a423-88bdfdd1fa7a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:06.325675 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d9655-005c-417e-a423-88bdfdd1fa7a None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:06.326470 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033d9655-005c-417e-a423-88bdfdd1fa7a None admin] RBAC: Adding query filter params (name=image) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:06.326776 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033d9655-005c-417e-a423-88bdfdd1fa7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:06.328145 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033d9655-005c-417e-a423-88bdfdd1fa7a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:06.337944 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 507/1032] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:26:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:26:06.343290 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d9655-005c-417e-a423-88bdfdd1fa7a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:06.349378 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:06.428175 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e17a87-d9b2-4ce8-b400-5161e3f1dca3 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:06.451632 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d9655-005c-417e-a423-88bdfdd1fa7a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:06.464799 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52356813-558a-4b1d-8f8c-df3f42658728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:06.465590 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52356813-558a-4b1d-8f8c-df3f42658728 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:06.465872 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52356813-558a-4b1d-8f8c-df3f42658728 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:06.466187 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52356813-558a-4b1d-8f8c-df3f42658728 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:06.476801 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52356813-558a-4b1d-8f8c-df3f42658728 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:06.478414 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52356813-558a-4b1d-8f8c-df3f42658728 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:06.490010 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f763a66-33b5-4fef-8bea-ab14eb3cb6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:06.490892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f763a66-33b5-4fef-8bea-ab14eb3cb6ab None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:06.491328 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f763a66-33b5-4fef-8bea-ab14eb3cb6ab None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:06.491774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f763a66-33b5-4fef-8bea-ab14eb3cb6ab None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:06.493172 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f763a66-33b5-4fef-8bea-ab14eb3cb6ab None admin] RBAC: Adding query filter params (type=image) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:06.493639 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f763a66-33b5-4fef-8bea-ab14eb3cb6ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:06.495179 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f763a66-33b5-4fef-8bea-ab14eb3cb6ab None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:06.496896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-52356813-558a-4b1d-8f8c-df3f42658728 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:26:06.496896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:26:06.496896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:26:06.496896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:26:06.496896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:26:06.496896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:26:06.496896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:26:06.496896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:26:06.496896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:26:06.496896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:26:06.496896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:26:06.496896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:26:06.496896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:26:06.496896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:26:06.496896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:26:06.497998 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:26:06.497998 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:26:06.497998 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:26:06.497998 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:26:06.497998 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:26:06.497998 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:26:06.497998 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:26:06.497998 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:26:06.497998 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:26:06.497998 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:26:06.497998 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:26:06.497998 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:26:06.497998 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:26:06.497998 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:26:06.497998 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:26:06.497998 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:26:06.497998 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:26:06.499256 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:26:06.499256 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:26:06.499256 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:26:06.499256 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:26:06.499256 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 526/1033] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:26:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:26:06.503657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 508/1034] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:26:06 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:26:06.503839 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52356813-558a-4b1d-8f8c-df3f42658728 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:06.509931 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f763a66-33b5-4fef-8bea-ab14eb3cb6ab None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:06.527493 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:06.590882 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:06.616972 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52356813-558a-4b1d-8f8c-df3f42658728 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:06.622252 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f763a66-33b5-4fef-8bea-ab14eb3cb6ab None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:06.656357 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada4f76-3475-44d4-b018-4cae9babaef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:06.656924 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab90ca-e753-4736-a514-74ed4e17eca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:06.657061 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada4f76-3475-44d4-b018-4cae9babaef8 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:06.657290 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada4f76-3475-44d4-b018-4cae9babaef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:06.657568 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada4f76-3475-44d4-b018-4cae9babaef8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:06.658373 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab90ca-e753-4736-a514-74ed4e17eca4 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:06.658483 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada4f76-3475-44d4-b018-4cae9babaef8 None admin] RBAC: Adding query filter params (interface=public, service_id=30e352631de44b40bf0b995472edeaf5, region_id=RegionOne) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:06.658694 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada4f76-3475-44d4-b018-4cae9babaef8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:06.659199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab90ca-e753-4736-a514-74ed4e17eca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:06.659721 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab90ca-e753-4736-a514-74ed4e17eca4 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:06.660058 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada4f76-3475-44d4-b018-4cae9babaef8 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:06.660877 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ab90ca-e753-4736-a514-74ed4e17eca4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:06.661507 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ab90ca-e753-4736-a514-74ed4e17eca4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:06.663220 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ab90ca-e753-4736-a514-74ed4e17eca4 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:06.665773 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 509/1035] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Jun 15 19:26:06 2022] GET /identity/v3/endpoints?service_id=30e352631de44b40bf0b995472edeaf5&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:26:06.671254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada4f76-3475-44d4-b018-4cae9babaef8 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:06.685236 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 527/1036] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:26:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:26:06.686483 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab90ca-e753-4736-a514-74ed4e17eca4 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:06.753023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:06.777301 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada4f76-3475-44d4-b018-4cae9babaef8 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:06.790938 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab90ca-e753-4736-a514-74ed4e17eca4 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:06.813634 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ffb87b-3ab9-4b97-a8f8-a414aa91848a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:06.814320 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ffb87b-3ab9-4b97-a8f8-a414aa91848a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:06.814605 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ffb87b-3ab9-4b97-a8f8-a414aa91848a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:06.814880 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ffb87b-3ab9-4b97-a8f8-a414aa91848a None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:06.815570 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ffb87b-3ab9-4b97-a8f8-a414aa91848a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:06.816928 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ffb87b-3ab9-4b97-a8f8-a414aa91848a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:06.823488 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 510/1037] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Jun 15 19:26:06 2022] GET /identity/v3/services => generated 1957 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:26:06.828606 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53387d-f762-4b32-a434-4ce717348e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:06.829687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53387d-f762-4b32-a434-4ce717348e83 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:06.830195 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53387d-f762-4b32-a434-4ce717348e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:06.830684 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53387d-f762-4b32-a434-4ce717348e83 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:06.834825 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:06.841309 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b53387d-f762-4b32-a434-4ce717348e83 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:06.843122 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b53387d-f762-4b32-a434-4ce717348e83 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:06.859492 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-1b53387d-f762-4b32-a434-4ce717348e83 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 19:26:06.859492 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:26:06.859492 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:26:06.859492 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:26:06.859492 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:26:06.859492 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:26:06.859492 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:26:06.859492 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:26:06.859492 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:26:06.859492 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:26:06.859492 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:26:06.859492 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:26:06.859492 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 19:26:06.859492 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 19:26:06.859492 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 19:26:06.859492 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 19:26:06.860590 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:26:06.860590 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:26:06.860590 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:26:06.860590 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:26:06.860590 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:26:06.860590 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:26:06.860590 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:26:06.860590 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:26:06.860590 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:26:06.860590 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:26:06.860590 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:26:06.860590 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:26:06.860590 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:26:06.860590 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:26:06.860590 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:26:06.860590 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:26:06.860590 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:26:06.860590 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:26:06.861553 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:26:06.861553 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:26:06.861553 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 19:26:06.861553 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 19:26:06.861553 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 19:26:06.861553 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 19:26:06.861553 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 19:26:06.861553 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 19:26:06.861553 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 19:26:06.861553 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:26:06.862153 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 528/1038] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Jun 15 19:26:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:26:06.867191 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ffb87b-3ab9-4b97-a8f8-a414aa91848a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:06.977201 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ffb87b-3ab9-4b97-a8f8-a414aa91848a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:07.015267 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c324d9-c04e-4742-b8b7-60c8f7d1fd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:07.016040 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c324d9-c04e-4742-b8b7-60c8f7d1fd0b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:07.016211 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c324d9-c04e-4742-b8b7-60c8f7d1fd0b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:07.016485 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c324d9-c04e-4742-b8b7-60c8f7d1fd0b None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:07.017319 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c324d9-c04e-4742-b8b7-60c8f7d1fd0b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:07.017622 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c324d9-c04e-4742-b8b7-60c8f7d1fd0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:07.019066 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c324d9-c04e-4742-b8b7-60c8f7d1fd0b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:07.032855 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:07.037558 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 511/1039] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Jun 15 19:26:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:07.043636 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53387d-f762-4b32-a434-4ce717348e83 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:07.144235 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53387d-f762-4b32-a434-4ce717348e83 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:07.178744 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604159f-40e7-4b6a-add8-e50b3bd24e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:07.179394 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604159f-40e7-4b6a-add8-e50b3bd24e7d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:07.179737 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604159f-40e7-4b6a-add8-e50b3bd24e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:07.180012 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604159f-40e7-4b6a-add8-e50b3bd24e7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:07.240869 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:07.249389 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0604159f-40e7-4b6a-add8-e50b3bd24e7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:07.250935 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0604159f-40e7-4b6a-add8-e50b3bd24e7d None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:07.363408 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 529/1040] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:26:07 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 19:26:07.364636 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c324d9-c04e-4742-b8b7-60c8f7d1fd0b None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:07.470667 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c324d9-c04e-4742-b8b7-60c8f7d1fd0b None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:07.518797 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8cd8e-6028-4956-9ad8-c382ff49dc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:07.519536 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8cd8e-6028-4956-9ad8-c382ff49dc17 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:07.519818 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8cd8e-6028-4956-9ad8-c382ff49dc17 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:07.520091 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8cd8e-6028-4956-9ad8-c382ff49dc17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:07.532778 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8cd8e-6028-4956-9ad8-c382ff49dc17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:07.534302 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8cd8e-6028-4956-9ad8-c382ff49dc17 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:07.554343 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-4fe8cd8e-6028-4956-9ad8-c382ff49dc17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:26:07.554343 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:26:07.554343 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:26:07.554343 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:26:07.554343 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:26:07.554343 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:26:07.554343 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:26:07.554343 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:26:07.554343 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:26:07.554343 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:26:07.554343 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:26:07.554343 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:26:07.554343 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:26:07.554343 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:26:07.554343 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:26:07.555433 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:26:07.555433 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:26:07.555433 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:26:07.555433 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:26:07.555433 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:26:07.555433 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:26:07.555433 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:26:07.555433 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:26:07.555433 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:26:07.555433 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:26:07.555433 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:26:07.555433 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:26:07.555433 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:26:07.555433 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:26:07.555433 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:26:07.555433 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:26:07.555433 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:26:07.556464 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:26:07.556464 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:26:07.556464 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:26:07.556464 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:26:07.557296 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 512/1041] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:26:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:26:07.561369 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604159f-40e7-4b6a-add8-e50b3bd24e7d None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:07.591566 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5164cf8-8520-48fd-a4f1-3817d31d0150 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:07.591864 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5164cf8-8520-48fd-a4f1-3817d31d0150 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:07.592148 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5164cf8-8520-48fd-a4f1-3817d31d0150 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:07.593679 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 513/1042] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 19:26:07.600010 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641c95b-9ec0-4fcf-b3b6-a0036d215897 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:07.600300 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641c95b-9ec0-4fcf-b3b6-a0036d215897 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:07.600580 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641c95b-9ec0-4fcf-b3b6-a0036d215897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:07.617737 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-5641c95b-9ec0-4fcf-b3b6-a0036d215897 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:26:07.618972 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-5641c95b-9ec0-4fcf-b3b6-a0036d215897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:26:07.660121 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:07.667711 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604159f-40e7-4b6a-add8-e50b3bd24e7d None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:07.668472 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:07.691103 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 514/1043] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:26:07 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 19:26:07.707158 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587865c5-c3ba-495b-83d6-1e71b5a3ecda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:07.707813 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587865c5-c3ba-495b-83d6-1e71b5a3ecda None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:07.708093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587865c5-c3ba-495b-83d6-1e71b5a3ecda None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:07.708365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587865c5-c3ba-495b-83d6-1e71b5a3ecda None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:07.709076 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587865c5-c3ba-495b-83d6-1e71b5a3ecda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:07.709378 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587865c5-c3ba-495b-83d6-1e71b5a3ecda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:07.711369 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587865c5-c3ba-495b-83d6-1e71b5a3ecda None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:07.715187 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641c95b-9ec0-4fcf-b3b6-a0036d215897 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:07.728466 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 530/1044] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:07.729190 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587865c5-c3ba-495b-83d6-1e71b5a3ecda None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:07.816722 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641c95b-9ec0-4fcf-b3b6-a0036d215897 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:07.838595 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587865c5-c3ba-495b-83d6-1e71b5a3ecda None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:07.853795 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b5dbb-a07a-44d9-bb63-4d1785a1df38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:07.854619 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b5dbb-a07a-44d9-bb63-4d1785a1df38 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:07.854900 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b5dbb-a07a-44d9-bb63-4d1785a1df38 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:07.855173 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b5dbb-a07a-44d9-bb63-4d1785a1df38 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:07.865657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59b5dbb-a07a-44d9-bb63-4d1785a1df38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:26:07.873341 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-a59b5dbb-a07a-44d9-bb63-4d1785a1df38 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:26:07.873341 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:26:07.873341 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:26:07.873341 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:26:07.873341 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:26:07.873341 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:26:07.873341 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:26:07.873341 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:26:07.873341 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:26:07.873341 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:26:07.873341 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:26:07.873341 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:26:07.873341 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:26:07.873341 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:26:07.873341 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:26:07.873341 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:26:07.874579 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:26:07.874579 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:26:07.874579 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:26:07.874579 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:26:07.874579 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:26:07.874579 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:26:07.874579 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:26:07.874579 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:26:07.874579 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:26:07.874579 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:26:07.874579 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:26:07.874579 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:26:07.874579 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:26:07.874579 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:26:07.874579 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:26:07.874579 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:26:07.879126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:26:07.879126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:26:07.879126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:26:07.879126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:26:07.879126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:26:07.879126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:26:07.879126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:26:07.879126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:26:07.879126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:26:07.879126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:26:07.879126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:26:07.879126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:26:07.879801 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd7d18-8d2d-4338-a11d-5b2b211a0f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:07.879801 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd7d18-8d2d-4338-a11d-5b2b211a0f31 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:07.879951 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 515/1045] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jun 15 19:26:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:26:07.880030 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd7d18-8d2d-4338-a11d-5b2b211a0f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:07.880030 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd7d18-8d2d-4338-a11d-5b2b211a0f31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:07.880300 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b5dbb-a07a-44d9-bb63-4d1785a1df38 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:07.886870 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd7d18-8d2d-4338-a11d-5b2b211a0f31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:07.888340 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd7d18-8d2d-4338-a11d-5b2b211a0f31 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:07.905164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-eddd7d18-8d2d-4338-a11d-5b2b211a0f31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:26:07.905164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:26:07.905164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:26:07.905164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:26:07.905164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:26:07.905164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:26:07.905164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:26:07.905164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:26:07.905164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:26:07.905164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:26:07.905164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:26:07.905164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:26:07.905164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 19:26:07.905164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 19:26:07.905164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 19:26:07.906073 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 19:26:07.906073 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:26:07.906073 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:26:07.906073 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:26:07.906073 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:26:07.906073 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:26:07.906073 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:26:07.906073 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:26:07.906073 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:26:07.906073 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:26:07.906073 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:26:07.906073 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:26:07.906073 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:26:07.906073 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:26:07.906073 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:26:07.906073 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:26:07.906073 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:26:07.907112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 19:26:07.907112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 19:26:07.907112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:26:07.907112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:26:07.907112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:26:07.907112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:26:07.907112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:26:07.907112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:26:07.907112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 531/1046] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:26:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:26:07.911969 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd7d18-8d2d-4338-a11d-5b2b211a0f31 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:07.929273 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:07.933002 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:07.994012 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b5dbb-a07a-44d9-bb63-4d1785a1df38 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:08.018601 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd7d18-8d2d-4338-a11d-5b2b211a0f31 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:08.033061 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c8557-569b-4738-b212-015617defa6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:08.033896 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c8557-569b-4738-b212-015617defa6d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:08.034227 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c8557-569b-4738-b212-015617defa6d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:08.034512 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c8557-569b-4738-b212-015617defa6d None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:08.035269 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880c8557-569b-4738-b212-015617defa6d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:08.035581 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880c8557-569b-4738-b212-015617defa6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:08.036958 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880c8557-569b-4738-b212-015617defa6d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:08.051024 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 516/1047] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:26:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:08.052041 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c8557-569b-4738-b212-015617defa6d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:08.059985 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5965ff6-31aa-4361-862b-1c873a0bb1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:08.061039 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5965ff6-31aa-4361-862b-1c873a0bb1f2 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:08.061039 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5965ff6-31aa-4361-862b-1c873a0bb1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:08.061314 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5965ff6-31aa-4361-862b-1c873a0bb1f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:08.062130 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5965ff6-31aa-4361-862b-1c873a0bb1f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:08.062450 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5965ff6-31aa-4361-862b-1c873a0bb1f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:08.063886 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5965ff6-31aa-4361-862b-1c873a0bb1f2 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:08.076132 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 532/1048] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jun 15 19:26:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:26:08.082323 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5965ff6-31aa-4361-862b-1c873a0bb1f2 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:08.159346 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c8557-569b-4738-b212-015617defa6d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:08.161616 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:08.161732 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:08.191378 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5965ff6-31aa-4361-862b-1c873a0bb1f2 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:08.196147 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca87e89-ec33-4058-91ab-7620b1d2d47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:08.197025 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca87e89-ec33-4058-91ab-7620b1d2d47d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:08.197312 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca87e89-ec33-4058-91ab-7620b1d2d47d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:08.197590 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca87e89-ec33-4058-91ab-7620b1d2d47d None admin] PATH_INFO: `/v3/users/885a5bf7e411482a8071202cc12ea06b` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:08.211657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca87e89-ec33-4058-91ab-7620b1d2d47d None admin] RBAC: Authorizing `identity:get_user(user_id=885a5bf7e411482a8071202cc12ea06b)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:08.213532 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca87e89-ec33-4058-91ab-7620b1d2d47d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:08.215256 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 517/1049] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:26:08 2022] GET /identity/v3/users/885a5bf7e411482a8071202cc12ea06b => generated 285 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:08.220781 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca87e89-ec33-4058-91ab-7620b1d2d47d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:08.227167 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b6ac0b-785e-495b-9022-9c13c8f1b05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:08.227821 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b6ac0b-785e-495b-9022-9c13c8f1b05e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:08.228106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b6ac0b-785e-495b-9022-9c13c8f1b05e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:08.228383 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b6ac0b-785e-495b-9022-9c13c8f1b05e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:08.229215 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b6ac0b-785e-495b-9022-9c13c8f1b05e None admin] RBAC: Adding query filter params (role.id=34c882c657e2470e9bbe75d07694392d, scope.project.id=7fb8cc0eb560465e972ffd45867ee73b, user.id=3e74ac44e5814122a3226ac023f3688a) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:08.229528 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b6ac0b-785e-495b-9022-9c13c8f1b05e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:08.230975 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b6ac0b-785e-495b-9022-9c13c8f1b05e None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:08.236805 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 533/1050] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:26:08 2022] GET /identity/v3/role_assignments?user.id=3e74ac44e5814122a3226ac023f3688a&scope.project.id=7fb8cc0eb560465e972ffd45867ee73b&role.id=34c882c657e2470e9bbe75d07694392d => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:26:08.345302 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca87e89-ec33-4058-91ab-7620b1d2d47d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:08.383983 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602e13e-6cbf-4d17-9236-a6214c03eeac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:08.384649 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602e13e-6cbf-4d17-9236-a6214c03eeac None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:08.384934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602e13e-6cbf-4d17-9236-a6214c03eeac None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:08.385204 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602e13e-6cbf-4d17-9236-a6214c03eeac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:08.418433 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:08.456926 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a602e13e-6cbf-4d17-9236-a6214c03eeac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:08.458395 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a602e13e-6cbf-4d17-9236-a6214c03eeac None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:08.560990 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 518/1051] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:26:08 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 15 19:26:08.562502 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b6ac0b-785e-495b-9022-9c13c8f1b05e None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:08.663150 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b6ac0b-785e-495b-9022-9c13c8f1b05e None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:08.699308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781eab1-b8b8-4300-901d-a7668ec4571d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:08.700177 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781eab1-b8b8-4300-901d-a7668ec4571d None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:08.700466 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781eab1-b8b8-4300-901d-a7668ec4571d None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:08.700744 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781eab1-b8b8-4300-901d-a7668ec4571d None admin] PATH_INFO: `/v3/projects/c337126c82a1412e93a98ed265833fbd` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:08.701938 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781eab1-b8b8-4300-901d-a7668ec4571d None admin] RBAC: Authorizing `identity:get_project(project_id=c337126c82a1412e93a98ed265833fbd)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:08.703413 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781eab1-b8b8-4300-901d-a7668ec4571d None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:08.705289 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 534/1052] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:26:08 2022] GET /identity/v3/projects/c337126c82a1412e93a98ed265833fbd => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:26:08.711126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602e13e-6cbf-4d17-9236-a6214c03eeac None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:08.773407 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:08.813497 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602e13e-6cbf-4d17-9236-a6214c03eeac None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:08.847596 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd77e5d6-3535-479e-8e1e-ad895244ea93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:08.848261 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77e5d6-3535-479e-8e1e-ad895244ea93 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:08.848547 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77e5d6-3535-479e-8e1e-ad895244ea93 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:08.848826 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77e5d6-3535-479e-8e1e-ad895244ea93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:08.849662 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd77e5d6-3535-479e-8e1e-ad895244ea93 None admin] RBAC: Adding query filter params (role.id=4f736313c26d4f8db48bbb2f346e50e4, scope.project.id=c337126c82a1412e93a98ed265833fbd, user.id=885a5bf7e411482a8071202cc12ea06b) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:08.849981 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd77e5d6-3535-479e-8e1e-ad895244ea93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:08.851482 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd77e5d6-3535-479e-8e1e-ad895244ea93 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:08.866132 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 519/1053] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:26:08 2022] GET /identity/v3/role_assignments?user.id=885a5bf7e411482a8071202cc12ea06b&scope.project.id=c337126c82a1412e93a98ed265833fbd&role.id=4f736313c26d4f8db48bbb2f346e50e4 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:09.021793 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a39038-efa7-4b55-9e3e-cde501b73454 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:09.022973 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a39038-efa7-4b55-9e3e-cde501b73454 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:09.023267 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a39038-efa7-4b55-9e3e-cde501b73454 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:09.025050 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 535/1054] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:26:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 19:26:09.029949 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138e547-f66e-48a0-a49b-1a3ba206fb19 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:09.030274 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138e547-f66e-48a0-a49b-1a3ba206fb19 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:09.030557 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138e547-f66e-48a0-a49b-1a3ba206fb19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:09.047313 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c138e547-f66e-48a0-a49b-1a3ba206fb19 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:26:09.048501 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c138e547-f66e-48a0-a49b-1a3ba206fb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:26:09.118910 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 520/1055] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:26:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:26:09.141316 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a39038-efa7-4b55-9e3e-cde501b73454 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:09.153459 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:09.245393 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a39038-efa7-4b55-9e3e-cde501b73454 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:09.279216 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49deeb09-d8a3-4828-acbe-8972d9d829ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:09.279917 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49deeb09-d8a3-4828-acbe-8972d9d829ec None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:09.280199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49deeb09-d8a3-4828-acbe-8972d9d829ec None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:09.280475 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49deeb09-d8a3-4828-acbe-8972d9d829ec None admin] PATH_INFO: `/v3/services/image` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:09.281121 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49deeb09-d8a3-4828-acbe-8972d9d829ec None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:09.282511 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49deeb09-d8a3-4828-acbe-8972d9d829ec None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:09.294670 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-49deeb09-d8a3-4828-acbe-8972d9d829ec None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 19:26:09.294670 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:26:09.294670 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 19:26:09.294670 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 19:26:09.294670 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 19:26:09.294670 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 19:26:09.294670 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 19:26:09.294670 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:26:09.294670 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 19:26:09.294670 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:26:09.294670 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 19:26:09.294670 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:26:09.294670 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:26:09.294670 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:26:09.294670 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:26:09.294670 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:26:09.294670 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:26:09.295729 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:26:09.295729 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:26:09.295729 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:26:09.295729 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:26:09.295729 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:26:09.295729 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:26:09.295729 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 19:26:09.295729 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 19:26:09.295729 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 19:26:09.295729 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 19:26:09.295729 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:26:09.295729 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:26:09.295729 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:26:09.295729 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:26:09.295729 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:26:09.295729 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:26:09.296601 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:26:09.296601 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:26:09.296601 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:26:09.296601 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:26:09.296601 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:26:09.296601 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:26:09.296601 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:26:09.296601 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:26:09.296601 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:26:09.296601 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:26:09.296601 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 19:26:09.296601 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:26:09.296601 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 19:26:09.296601 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:26:09.298185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 536/1056] 10.222.0.15 () {56 vars in 1105 bytes} [Wed Jun 15 19:26:09 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:26:09.301796 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138e547-f66e-48a0-a49b-1a3ba206fb19 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:09.368223 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:09.416229 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138e547-f66e-48a0-a49b-1a3ba206fb19 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:09.454173 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a489f7e-5535-4145-9d4e-60606f3bd735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:09.454824 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a489f7e-5535-4145-9d4e-60606f3bd735 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:09.455107 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a489f7e-5535-4145-9d4e-60606f3bd735 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:09.455384 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a489f7e-5535-4145-9d4e-60606f3bd735 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:09.456090 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a489f7e-5535-4145-9d4e-60606f3bd735 None admin] RBAC: Adding query filter params (name=image) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:09.456394 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a489f7e-5535-4145-9d4e-60606f3bd735 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:09.457761 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a489f7e-5535-4145-9d4e-60606f3bd735 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:09.467793 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 521/1057] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:26:09 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:09.468923 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49deeb09-d8a3-4828-acbe-8972d9d829ec None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:09.538679 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:09.570707 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49deeb09-d8a3-4828-acbe-8972d9d829ec None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:09.603380 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d6a94-5360-4f5b-8fd9-ce0782bf06b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:09.604043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d6a94-5360-4f5b-8fd9-ce0782bf06b3 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:09.604329 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d6a94-5360-4f5b-8fd9-ce0782bf06b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:09.604607 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d6a94-5360-4f5b-8fd9-ce0782bf06b3 None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:09.605310 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d6a94-5360-4f5b-8fd9-ce0782bf06b3 None admin] RBAC: Adding query filter params (type=image) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:09.605617 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d6a94-5360-4f5b-8fd9-ce0782bf06b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:09.607001 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d6a94-5360-4f5b-8fd9-ce0782bf06b3 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:09.616775 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 537/1058] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:26:09 2022] GET /identity/v3/services?type=image => generated 350 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:09.618342 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a489f7e-5535-4145-9d4e-60606f3bd735 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:09.717767 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a489f7e-5535-4145-9d4e-60606f3bd735 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:09.740219 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:09.760355 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d030ae3f-2783-41bd-b9d3-d0d6ffb2ea59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:09.761021 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030ae3f-2783-41bd-b9d3-d0d6ffb2ea59 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:09.761303 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030ae3f-2783-41bd-b9d3-d0d6ffb2ea59 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:09.761577 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030ae3f-2783-41bd-b9d3-d0d6ffb2ea59 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:09.762368 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d030ae3f-2783-41bd-b9d3-d0d6ffb2ea59 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:09.763744 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d030ae3f-2783-41bd-b9d3-d0d6ffb2ea59 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:09.781677 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 522/1059] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Jun 15 19:26:09 2022] POST /identity/v3/endpoints => generated 327 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:26:10.360926 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1f45b-039a-4b86-8e30-6c5dea02bf2e None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:10.361433 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1f45b-039a-4b86-8e30-6c5dea02bf2e None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:10.361538 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1f45b-039a-4b86-8e30-6c5dea02bf2e None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:10.362916 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 538/1060] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:26:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:26:10.368862 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf183476-ef64-49f5-859f-4020f559646e None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:10.369290 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf183476-ef64-49f5-859f-4020f559646e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:10.369464 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf183476-ef64-49f5-859f-4020f559646e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:10.388213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-bf183476-ef64-49f5-859f-4020f559646e None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:26:10.389415 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-bf183476-ef64-49f5-859f-4020f559646e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:26:10.468560 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 523/1061] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:26:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:26:10.490670 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c1f45b-039a-4b86-8e30-6c5dea02bf2e None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:10.594377 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c1f45b-039a-4b86-8e30-6c5dea02bf2e None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:10.629031 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6b497-bb85-40bc-b47f-b07e9806681a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:10.629845 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6b497-bb85-40bc-b47f-b07e9806681a None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:10.630155 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6b497-bb85-40bc-b47f-b07e9806681a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:10.630434 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6b497-bb85-40bc-b47f-b07e9806681a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:10.633358 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:10.642179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d6b497-bb85-40bc-b47f-b07e9806681a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:26:10.650308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-89d6b497-bb85-40bc-b47f-b07e9806681a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 19:26:10.650308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:26:10.650308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:26:10.650308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:26:10.650308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:26:10.650308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:26:10.650308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:26:10.650308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:26:10.650308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:26:10.650308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:26:10.650308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:26:10.650308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:26:10.650308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:26:10.650308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:26:10.650308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:26:10.650308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:26:10.651275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:26:10.651275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:26:10.651275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:26:10.651275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:26:10.651275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:26:10.651275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:26:10.651275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:26:10.651275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:26:10.651275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:26:10.651275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:26:10.651275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:26:10.651275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:26:10.651275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:26:10.651275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:26:10.651275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:26:10.651275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:26:10.652127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:26:10.652127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:26:10.652127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:26:10.652127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:26:10.652127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:26:10.652127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:26:10.652127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:26:10.652127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:26:10.652127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:26:10.652127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:26:10.652127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 19:26:10.652127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:26:10.653427 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 539/1062] 10.222.0.15 () {56 vars in 1120 bytes} [Wed Jun 15 19:26:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:26:10.657607 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf183476-ef64-49f5-859f-4020f559646e None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:10.759375 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf183476-ef64-49f5-859f-4020f559646e None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:10.760894 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:10.796075 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d1675d-efb6-405d-8119-059ccb19c6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:10.796748 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1675d-efb6-405d-8119-059ccb19c6a9 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:10.797029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1675d-efb6-405d-8119-059ccb19c6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:10.797302 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1675d-efb6-405d-8119-059ccb19c6a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:10.798052 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d1675d-efb6-405d-8119-059ccb19c6a9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:10.798391 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d1675d-efb6-405d-8119-059ccb19c6a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:10.799783 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d1675d-efb6-405d-8119-059ccb19c6a9 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:10.812900 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 524/1063] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Jun 15 19:26:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:26:10.814168 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6b497-bb85-40bc-b47f-b07e9806681a None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:10.921146 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6b497-bb85-40bc-b47f-b07e9806681a None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:10.956042 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1164d81f-5e54-43fb-8418-89d02f8b265e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:10.956839 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164d81f-5e54-43fb-8418-89d02f8b265e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:10.957122 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164d81f-5e54-43fb-8418-89d02f8b265e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:10.957397 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164d81f-5e54-43fb-8418-89d02f8b265e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:10.964901 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a3c1f-d3b5-49fd-86f7-996cf8372a3b None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:10.965195 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a3c1f-d3b5-49fd-86f7-996cf8372a3b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:10.965474 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a3c1f-d3b5-49fd-86f7-996cf8372a3b None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:10.966894 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 525/1064] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:26:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:26:10.970476 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1164d81f-5e54-43fb-8418-89d02f8b265e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:10.971932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1164d81f-5e54-43fb-8418-89d02f8b265e None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:10.973189 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4abfea-81f6-463b-8bcd-830f07f90c07 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:10.973941 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4abfea-81f6-463b-8bcd-830f07f90c07 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:10.974864 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4abfea-81f6-463b-8bcd-830f07f90c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:10.989361 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-1164d81f-5e54-43fb-8418-89d02f8b265e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:26:10.989361 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:26:10.989361 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:26:10.989361 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:26:10.989361 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:26:10.989361 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:26:10.989361 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:26:10.989361 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:26:10.989361 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:26:10.989361 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:26:10.989361 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:26:10.989361 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:26:10.989361 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:26:10.989361 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:26:10.989361 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:26:10.990360 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:26:10.990360 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:26:10.990360 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:26:10.990360 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:26:10.990360 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:26:10.990360 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:26:10.990360 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:26:10.990360 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:26:10.990360 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:26:10.990360 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:26:10.990360 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:26:10.990360 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:26:10.990360 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:26:10.990360 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:26:10.990360 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:26:10.990360 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:26:10.990360 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:26:10.991266 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:26:10.991266 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:26:10.991266 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:26:10.991266 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:26:10.991266 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 540/1065] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:26:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:26:10.994312 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-9a4abfea-81f6-463b-8bcd-830f07f90c07 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:26:10.995722 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-9a4abfea-81f6-463b-8bcd-830f07f90c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:26:10.996570 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1164d81f-5e54-43fb-8418-89d02f8b265e None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:11.048042 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:11.067732 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 526/1066] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:26:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 19:26:11.091860 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4abfea-81f6-463b-8bcd-830f07f90c07 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:11.100972 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1164d81f-5e54-43fb-8418-89d02f8b265e None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:11.139289 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c187026-6889-4cab-af09-7a72b7758839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:11.139855 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c187026-6889-4cab-af09-7a72b7758839 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:11.140143 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c187026-6889-4cab-af09-7a72b7758839 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:11.140681 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c187026-6889-4cab-af09-7a72b7758839 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:11.141127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c187026-6889-4cab-af09-7a72b7758839 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:11.141431 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c187026-6889-4cab-af09-7a72b7758839 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:11.142880 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c187026-6889-4cab-af09-7a72b7758839 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:11.154655 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 541/1067] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:26:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:26:11.159875 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c187026-6889-4cab-af09-7a72b7758839 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:11.192479 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:11.199633 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4abfea-81f6-463b-8bcd-830f07f90c07 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:11.234644 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742bf4ec-95af-41b8-81a7-2f39b449de9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:11.235573 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742bf4ec-95af-41b8-81a7-2f39b449de9f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:11.235859 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742bf4ec-95af-41b8-81a7-2f39b449de9f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:11.236131 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742bf4ec-95af-41b8-81a7-2f39b449de9f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:11.246859 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742bf4ec-95af-41b8-81a7-2f39b449de9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:26:11.254478 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-742bf4ec-95af-41b8-81a7-2f39b449de9f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:26:11.254478 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:26:11.254478 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:26:11.254478 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:26:11.254478 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:26:11.254478 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:26:11.254478 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:26:11.254478 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:26:11.254478 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:26:11.254478 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:26:11.254478 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:26:11.254478 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:26:11.254478 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:26:11.254478 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:26:11.254478 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:26:11.254478 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:26:11.255411 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:26:11.255411 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:26:11.255411 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:26:11.255411 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:26:11.255411 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:26:11.255411 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:26:11.255411 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:26:11.255411 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:26:11.255411 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:26:11.255411 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:26:11.255411 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:26:11.255411 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:26:11.255411 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:26:11.255411 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:26:11.255411 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:26:11.255411 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:26:11.256248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:26:11.256248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:26:11.256248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:26:11.256248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:26:11.256248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:26:11.256248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:26:11.256248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:26:11.256248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:26:11.256248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:26:11.256248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:26:11.256248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:26:11.256248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:26:11.261171 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 527/1068] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jun 15 19:26:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:26:11.261888 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742bf4ec-95af-41b8-81a7-2f39b449de9f None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:11.270901 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c187026-6889-4cab-af09-7a72b7758839 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:11.280828 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:11.308731 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eff0a78-e2c8-468d-b624-a0cc76a8cef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:11.309522 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff0a78-e2c8-468d-b624-a0cc76a8cef4 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:11.309809 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff0a78-e2c8-468d-b624-a0cc76a8cef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:11.310104 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff0a78-e2c8-468d-b624-a0cc76a8cef4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:11.317123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eff0a78-e2c8-468d-b624-a0cc76a8cef4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:11.318596 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eff0a78-e2c8-468d-b624-a0cc76a8cef4 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:11.334189 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-0eff0a78-e2c8-468d-b624-a0cc76a8cef4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 19:26:11.334189 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:26:11.334189 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:26:11.334189 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:26:11.334189 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:26:11.334189 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:26:11.334189 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:26:11.334189 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:26:11.334189 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:26:11.334189 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:26:11.334189 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:26:11.334189 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:26:11.334189 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 19:26:11.334189 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 19:26:11.334189 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 19:26:11.334189 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 19:26:11.335182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:26:11.335182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:26:11.335182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:26:11.335182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:26:11.335182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:26:11.335182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:26:11.335182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:26:11.335182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:26:11.335182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:26:11.335182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:26:11.335182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:26:11.335182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:26:11.335182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:26:11.335182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:26:11.335182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:26:11.335182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:26:11.335182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:26:11.335182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:26:11.336165 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:26:11.336165 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:26:11.336165 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 19:26:11.336165 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 19:26:11.336165 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 19:26:11.336165 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 19:26:11.336165 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 19:26:11.336165 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 19:26:11.336165 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 19:26:11.336165 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:26:11.336165 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 542/1069] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Jun 15 19:26:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:26:11.340879 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eff0a78-e2c8-468d-b624-a0cc76a8cef4 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:11.384591 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742bf4ec-95af-41b8-81a7-2f39b449de9f None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:11.419259 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff0c349-223f-470e-9dbe-4a30416a5579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:11.419926 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff0c349-223f-470e-9dbe-4a30416a5579 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:11.420203 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff0c349-223f-470e-9dbe-4a30416a5579 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:11.420472 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff0c349-223f-470e-9dbe-4a30416a5579 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:11.421214 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff0c349-223f-470e-9dbe-4a30416a5579 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:11.421517 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff0c349-223f-470e-9dbe-4a30416a5579 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:11.423117 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff0c349-223f-470e-9dbe-4a30416a5579 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:11.432018 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 528/1070] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:26:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:26:11.437528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff0c349-223f-470e-9dbe-4a30416a5579 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:11.454069 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eff0a78-e2c8-468d-b624-a0cc76a8cef4 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:11.457102 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:11.489728 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16abef48-9195-4b2e-9c6c-24c131458240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:11.490430 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abef48-9195-4b2e-9c6c-24c131458240 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:11.490712 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abef48-9195-4b2e-9c6c-24c131458240 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:11.490987 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abef48-9195-4b2e-9c6c-24c131458240 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:11.491888 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16abef48-9195-4b2e-9c6c-24c131458240 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:11.492700 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16abef48-9195-4b2e-9c6c-24c131458240 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:11.494142 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16abef48-9195-4b2e-9c6c-24c131458240 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:11.512048 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 543/1071] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Jun 15 19:26:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:26:11.517407 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16abef48-9195-4b2e-9c6c-24c131458240 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:11.544260 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:11.551395 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff0c349-223f-470e-9dbe-4a30416a5579 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:11.588891 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09a719-c544-46b2-95e5-c2501e5194e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:11.589766 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09a719-c544-46b2-95e5-c2501e5194e0 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:11.590046 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09a719-c544-46b2-95e5-c2501e5194e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:11.590349 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09a719-c544-46b2-95e5-c2501e5194e0 None admin] PATH_INFO: `/v3/users/885a5bf7e411482a8071202cc12ea06b` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:11.591549 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd09a719-c544-46b2-95e5-c2501e5194e0 None admin] RBAC: Authorizing `identity:get_user(user_id=885a5bf7e411482a8071202cc12ea06b)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:11.592979 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd09a719-c544-46b2-95e5-c2501e5194e0 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:11.594659 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 529/1072] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:26:11 2022] GET /identity/v3/users/885a5bf7e411482a8071202cc12ea06b => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:26:11.600190 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09a719-c544-46b2-95e5-c2501e5194e0 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:11.621006 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16abef48-9195-4b2e-9c6c-24c131458240 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:11.657909 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366335c4-9b64-47fb-9550-9d8fa77b4896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:11.658727 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366335c4-9b64-47fb-9550-9d8fa77b4896 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:11.659008 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366335c4-9b64-47fb-9550-9d8fa77b4896 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:11.659279 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366335c4-9b64-47fb-9550-9d8fa77b4896 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:11.670388 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366335c4-9b64-47fb-9550-9d8fa77b4896 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:11.672744 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366335c4-9b64-47fb-9550-9d8fa77b4896 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:11.687382 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:11.691866 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-366335c4-9b64-47fb-9550-9d8fa77b4896 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:26:11.691866 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:26:11.691866 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:26:11.691866 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:26:11.691866 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:26:11.691866 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:26:11.691866 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:26:11.691866 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:26:11.691866 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:26:11.691866 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:26:11.691866 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:26:11.691866 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:26:11.691866 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:26:11.691866 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:26:11.691866 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:26:11.692933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:26:11.692933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:26:11.692933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:26:11.692933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:26:11.692933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:26:11.692933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:26:11.692933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:26:11.692933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:26:11.692933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:26:11.692933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:26:11.692933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:26:11.692933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:26:11.692933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:26:11.692933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:26:11.692933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:26:11.692933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:26:11.692933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:26:11.695636 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:26:11.695636 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:26:11.695636 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:26:11.695636 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:26:11.695636 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 544/1073] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:26:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:26:11.699067 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366335c4-9b64-47fb-9550-9d8fa77b4896 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:11.709456 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09a719-c544-46b2-95e5-c2501e5194e0 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:11.745326 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb921677-2d5c-4fcb-a07a-ed220a4f3a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:11.746351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb921677-2d5c-4fcb-a07a-ed220a4f3a60 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:11.746733 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb921677-2d5c-4fcb-a07a-ed220a4f3a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:11.747085 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb921677-2d5c-4fcb-a07a-ed220a4f3a60 None admin] PATH_INFO: `/v3/projects/c337126c82a1412e93a98ed265833fbd` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:11.748358 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb921677-2d5c-4fcb-a07a-ed220a4f3a60 None admin] RBAC: Authorizing `identity:get_project(project_id=c337126c82a1412e93a98ed265833fbd)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:11.749797 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb921677-2d5c-4fcb-a07a-ed220a4f3a60 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:11.756226 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 530/1074] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:26:11 2022] GET /identity/v3/projects/c337126c82a1412e93a98ed265833fbd => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:26:11.757119 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb921677-2d5c-4fcb-a07a-ed220a4f3a60 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:11.787615 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:11.811895 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366335c4-9b64-47fb-9550-9d8fa77b4896 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:11.852106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a33b0a-07a7-4e5c-a186-41a4c3e6d151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:11.852766 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a33b0a-07a7-4e5c-a186-41a4c3e6d151 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:11.853045 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a33b0a-07a7-4e5c-a186-41a4c3e6d151 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:11.853316 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a33b0a-07a7-4e5c-a186-41a4c3e6d151 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:11.854034 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a33b0a-07a7-4e5c-a186-41a4c3e6d151 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:11.854366 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a33b0a-07a7-4e5c-a186-41a4c3e6d151 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:11.855724 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a33b0a-07a7-4e5c-a186-41a4c3e6d151 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:11.862663 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb921677-2d5c-4fcb-a07a-ed220a4f3a60 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:11.868790 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 545/1075] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:26:11.874640 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a33b0a-07a7-4e5c-a186-41a4c3e6d151 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:11.899487 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3e0c1-b469-45e0-a982-237b6bb968bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:11.900666 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3e0c1-b469-45e0-a982-237b6bb968bc None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:11.901099 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3e0c1-b469-45e0-a982-237b6bb968bc None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:11.901523 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3e0c1-b469-45e0-a982-237b6bb968bc None admin] PATH_INFO: `/v3/projects/c337126c82a1412e93a98ed265833fbd/users/885a5bf7e411482a8071202cc12ea06b/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:11.903818 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae3e0c1-b469-45e0-a982-237b6bb968bc None admin] RBAC: Authorizing `identity:create_grant(project_id=c337126c82a1412e93a98ed265833fbd, user_id=885a5bf7e411482a8071202cc12ea06b, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:11.905499 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae3e0c1-b469-45e0-a982-237b6bb968bc None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:11.922622 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 531/1076] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jun 15 19:26:11 2022] PUT /identity/v3/projects/c337126c82a1412e93a98ed265833fbd/users/885a5bf7e411482a8071202cc12ea06b/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:26:11.983467 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18021b-c8ed-42af-aafe-4c84a0cf9f20 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:11.983652 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18021b-c8ed-42af-aafe-4c84a0cf9f20 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:11.983938 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18021b-c8ed-42af-aafe-4c84a0cf9f20 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:11.985365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 532/1077] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 19:26:11.991135 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff4d48a-b8c5-4366-a913-4eb4d8a88606 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:11.991429 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff4d48a-b8c5-4366-a913-4eb4d8a88606 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:11.991714 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff4d48a-b8c5-4366-a913-4eb4d8a88606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:11.996059 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a33b0a-07a7-4e5c-a186-41a4c3e6d151 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:12.010774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-bff4d48a-b8c5-4366-a913-4eb4d8a88606 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:26:12.012021 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-bff4d48a-b8c5-4366-a913-4eb4d8a88606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:26:12.021159 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:12.037765 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6f6709-4d18-4c26-8bac-3a301633c073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:12.040974 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f6709-4d18-4c26-8bac-3a301633c073 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:12.041257 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f6709-4d18-4c26-8bac-3a301633c073 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:12.041545 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f6709-4d18-4c26-8bac-3a301633c073 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:12.049852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6f6709-4d18-4c26-8bac-3a301633c073 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:12.051362 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6f6709-4d18-4c26-8bac-3a301633c073 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:12.053370 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:12.069810 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-7c6f6709-4d18-4c26-8bac-3a301633c073 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:26:12.069810 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:26:12.069810 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:26:12.069810 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:26:12.069810 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:26:12.069810 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:26:12.069810 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:26:12.069810 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:26:12.069810 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:26:12.069810 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:26:12.069810 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:26:12.069810 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:26:12.069810 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 19:26:12.069810 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 19:26:12.069810 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 19:26:12.070959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 19:26:12.070959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:26:12.070959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:26:12.070959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:26:12.070959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:26:12.070959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:26:12.070959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:26:12.070959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:26:12.070959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:26:12.070959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:26:12.070959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:26:12.070959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:26:12.070959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:26:12.070959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:26:12.070959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:26:12.070959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:26:12.070959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:26:12.071852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 19:26:12.071852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 19:26:12.071852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:26:12.071852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:26:12.071852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:26:12.071852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:26:12.071852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:26:12.071852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:26:12.071852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 546/1078] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:26:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:26:12.077617 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6f6709-4d18-4c26-8bac-3a301633c073 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:12.101129 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 533/1079] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:26:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 19:26:12.123894 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff4d48a-b8c5-4366-a913-4eb4d8a88606 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:12.193327 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6f6709-4d18-4c26-8bac-3a301633c073 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:12.227962 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff4d48a-b8c5-4366-a913-4eb4d8a88606 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:12.233847 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cdf681-8e9f-4fe7-ab7b-4270a14eddc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:12.234869 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdf681-8e9f-4fe7-ab7b-4270a14eddc8 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:12.235162 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdf681-8e9f-4fe7-ab7b-4270a14eddc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:12.235445 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdf681-8e9f-4fe7-ab7b-4270a14eddc8 None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:12.236670 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cdf681-8e9f-4fe7-ab7b-4270a14eddc8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:12.236989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cdf681-8e9f-4fe7-ab7b-4270a14eddc8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:12.238822 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cdf681-8e9f-4fe7-ab7b-4270a14eddc8 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:12.251857 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 547/1080] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jun 15 19:26:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:26:12.257271 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cdf681-8e9f-4fe7-ab7b-4270a14eddc8 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:12.268399 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fdff97-01f7-4412-855f-47673c88b700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:12.269053 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fdff97-01f7-4412-855f-47673c88b700 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:12.269549 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fdff97-01f7-4412-855f-47673c88b700 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:12.269760 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fdff97-01f7-4412-855f-47673c88b700 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:12.284247 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:12.316558 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:12.353955 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fdff97-01f7-4412-855f-47673c88b700 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:12.355472 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fdff97-01f7-4412-855f-47673c88b700 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:12.362309 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cdf681-8e9f-4fe7-ab7b-4270a14eddc8 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:12.398928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba19225-4d03-40fe-803f-8555c48adda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:12.399818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba19225-4d03-40fe-803f-8555c48adda8 None admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:12.400308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba19225-4d03-40fe-803f-8555c48adda8 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:12.400437 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba19225-4d03-40fe-803f-8555c48adda8 None admin] PATH_INFO: `/v3/projects/7fb8cc0eb560465e972ffd45867ee73b/users/3e74ac44e5814122a3226ac023f3688a/roles/34c882c657e2470e9bbe75d07694392d` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:12.402653 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba19225-4d03-40fe-803f-8555c48adda8 None admin] RBAC: Authorizing `identity:create_grant(project_id=7fb8cc0eb560465e972ffd45867ee73b, user_id=3e74ac44e5814122a3226ac023f3688a, role_id=34c882c657e2470e9bbe75d07694392d)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:12.404076 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba19225-4d03-40fe-803f-8555c48adda8 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:12.417654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 548/1081] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jun 15 19:26:12 2022] PUT /identity/v3/projects/7fb8cc0eb560465e972ffd45867ee73b/users/3e74ac44e5814122a3226ac023f3688a/roles/34c882c657e2470e9bbe75d07694392d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:26:12.472209 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 534/1082] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:26:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 15 19:26:12.478947 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba19225-4d03-40fe-803f-8555c48adda8 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:12.594010 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba19225-4d03-40fe-803f-8555c48adda8 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:12.611026 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:12.630231 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3841c630-6747-4b64-b917-4a885da4166e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:12.631034 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841c630-6747-4b64-b917-4a885da4166e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:12.631320 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841c630-6747-4b64-b917-4a885da4166e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:12.631599 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841c630-6747-4b64-b917-4a885da4166e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:12.643160 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3841c630-6747-4b64-b917-4a885da4166e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:12.644606 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3841c630-6747-4b64-b917-4a885da4166e None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:12.661630 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-3841c630-6747-4b64-b917-4a885da4166e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:26:12.661630 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:26:12.661630 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:26:12.661630 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:26:12.661630 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:26:12.661630 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:26:12.661630 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:26:12.661630 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:26:12.661630 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:26:12.661630 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:26:12.661630 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:26:12.661630 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:26:12.661630 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:26:12.661630 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:26:12.661630 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:26:12.662829 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:26:12.662829 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:26:12.662829 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:26:12.662829 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:26:12.662829 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:26:12.662829 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:26:12.662829 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:26:12.662829 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:26:12.662829 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:26:12.662829 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:26:12.662829 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:26:12.662829 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:26:12.662829 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:26:12.662829 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:26:12.662829 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:26:12.662829 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:26:12.662829 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:26:12.663774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:26:12.663774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:26:12.663774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:26:12.663774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:26:12.666146 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 549/1083] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:26:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:26:12.669058 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fdff97-01f7-4412-855f-47673c88b700 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:12.713502 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:12.778653 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fdff97-01f7-4412-855f-47673c88b700 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:12.813838 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0586d6-01c1-44da-af49-8af21805b9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:12.814746 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0586d6-01c1-44da-af49-8af21805b9cd None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:12.815191 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0586d6-01c1-44da-af49-8af21805b9cd None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:12.815607 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0586d6-01c1-44da-af49-8af21805b9cd None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:12.816466 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0586d6-01c1-44da-af49-8af21805b9cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:12.816927 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0586d6-01c1-44da-af49-8af21805b9cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:12.818450 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0586d6-01c1-44da-af49-8af21805b9cd None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:12.830533 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 535/1084] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:26:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:14.048039 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5727b8-5d8d-4118-9914-557446e5eca0 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:14.048601 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5727b8-5d8d-4118-9914-557446e5eca0 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:14.048601 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5727b8-5d8d-4118-9914-557446e5eca0 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:14.049945 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 550/1085] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:26:14.056234 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ed389-4d57-40d9-a786-0df5284d17c3 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:14.056636 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ed389-4d57-40d9-a786-0df5284d17c3 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:14.056804 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ed389-4d57-40d9-a786-0df5284d17c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:14.073260 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-f38ed389-4d57-40d9-a786-0df5284d17c3 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:26:14.074468 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f38ed389-4d57-40d9-a786-0df5284d17c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:26:14.140673 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 536/1086] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:26:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:26:14.164035 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5727b8-5d8d-4118-9914-557446e5eca0 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:14.265002 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5727b8-5d8d-4118-9914-557446e5eca0 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:14.297459 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ddf6c4-950c-4746-b5a2-6f0557edc443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:14.298320 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ddf6c4-950c-4746-b5a2-6f0557edc443 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:14.298555 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ddf6c4-950c-4746-b5a2-6f0557edc443 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:14.298831 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ddf6c4-950c-4746-b5a2-6f0557edc443 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:14.309179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ddf6c4-950c-4746-b5a2-6f0557edc443 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:26:14.316724 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-18ddf6c4-950c-4746-b5a2-6f0557edc443 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:26:14.316724 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:26:14.316724 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:26:14.316724 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:26:14.316724 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:26:14.316724 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:26:14.316724 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:26:14.316724 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:26:14.316724 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:26:14.316724 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:26:14.316724 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:26:14.316724 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:26:14.316724 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:26:14.316724 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:26:14.316724 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:26:14.316724 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:26:14.317804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:26:14.317804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:26:14.317804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:26:14.317804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:26:14.317804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:26:14.317804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:26:14.317804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:26:14.317804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:26:14.317804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:26:14.317804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:26:14.317804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:26:14.317804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:26:14.317804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:26:14.317804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:26:14.317804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:26:14.317804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:26:14.318705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:26:14.318705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:26:14.318705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:26:14.318705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:26:14.318705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:26:14.318705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:26:14.318705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:26:14.318705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:26:14.318705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:26:14.318705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:26:14.318705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:26:14.318705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:26:14.318705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 551/1087] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jun 15 19:26:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:26:14.324534 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ed389-4d57-40d9-a786-0df5284d17c3 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:14.358304 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:14.430852 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ed389-4d57-40d9-a786-0df5284d17c3 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:14.469819 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556aef02-bb16-4953-aa35-042f02cb3ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:14.470555 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556aef02-bb16-4953-aa35-042f02cb3ffd None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:14.470837 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556aef02-bb16-4953-aa35-042f02cb3ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:14.471112 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556aef02-bb16-4953-aa35-042f02cb3ffd None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:14.471883 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556aef02-bb16-4953-aa35-042f02cb3ffd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:14.472192 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556aef02-bb16-4953-aa35-042f02cb3ffd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:14.473618 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556aef02-bb16-4953-aa35-042f02cb3ffd None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:14.489909 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 537/1088] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:26:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:14.496728 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ddf6c4-950c-4746-b5a2-6f0557edc443 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:14.526352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:14.530580 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b5671-7d5b-4b4b-b6a5-201d6f07ee50 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:14.530872 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b5671-7d5b-4b4b-b6a5-201d6f07ee50 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:14.531156 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b5671-7d5b-4b4b-b6a5-201d6f07ee50 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:14.532760 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 538/1089] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 19:26:14.537316 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b3c50-792a-4463-a30b-8da91e50699f None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:14.537606 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b3c50-792a-4463-a30b-8da91e50699f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:14.537885 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b3c50-792a-4463-a30b-8da91e50699f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:14.555582 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-ad9b3c50-792a-4463-a30b-8da91e50699f None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:26:14.556706 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ad9b3c50-792a-4463-a30b-8da91e50699f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:26:14.595676 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ddf6c4-950c-4746-b5a2-6f0557edc443 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:14.623310 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 539/1090] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:26:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 19:26:14.629119 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff2c4fd-7c24-4a87-89a6-adbeec9927da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:14.629985 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2c4fd-7c24-4a87-89a6-adbeec9927da None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:14.630290 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2c4fd-7c24-4a87-89a6-adbeec9927da None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:14.630564 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2c4fd-7c24-4a87-89a6-adbeec9927da None admin] PATH_INFO: `/v3/users/885a5bf7e411482a8071202cc12ea06b` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:14.631727 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff2c4fd-7c24-4a87-89a6-adbeec9927da None admin] RBAC: Authorizing `identity:get_user(user_id=885a5bf7e411482a8071202cc12ea06b)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:14.633885 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff2c4fd-7c24-4a87-89a6-adbeec9927da None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:14.640041 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 552/1091] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:26:14 2022] GET /identity/v3/users/885a5bf7e411482a8071202cc12ea06b => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:14.641104 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9b3c50-792a-4463-a30b-8da91e50699f None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:14.649235 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff2c4fd-7c24-4a87-89a6-adbeec9927da None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:14.665797 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:14.740398 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9b3c50-792a-4463-a30b-8da91e50699f None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:14.745585 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:14.749685 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff2c4fd-7c24-4a87-89a6-adbeec9927da None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:14.775669 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4403bdfd-5c84-4061-be52-fc9da21893f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:14.776321 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403bdfd-5c84-4061-be52-fc9da21893f4 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:14.776604 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403bdfd-5c84-4061-be52-fc9da21893f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:14.776888 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403bdfd-5c84-4061-be52-fc9da21893f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:14.786216 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75072c8-5b8e-4e85-baaa-0fd16ef3c065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:14.787015 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75072c8-5b8e-4e85-baaa-0fd16ef3c065 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:14.787299 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75072c8-5b8e-4e85-baaa-0fd16ef3c065 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:14.787622 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75072c8-5b8e-4e85-baaa-0fd16ef3c065 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:14.797634 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75072c8-5b8e-4e85-baaa-0fd16ef3c065 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:26:14.806487 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-a75072c8-5b8e-4e85-baaa-0fd16ef3c065 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 19:26:14.806487 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:26:14.806487 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:26:14.806487 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:26:14.806487 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:26:14.806487 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:26:14.806487 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:26:14.806487 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:26:14.806487 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:26:14.806487 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:26:14.806487 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:26:14.806487 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:26:14.806487 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:26:14.806487 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:26:14.806487 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:26:14.806487 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:26:14.807495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:26:14.807495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:26:14.807495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:26:14.807495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:26:14.807495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:26:14.807495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:26:14.807495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:26:14.807495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:26:14.807495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:26:14.807495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:26:14.807495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:26:14.807495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:26:14.807495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:26:14.807495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:26:14.807495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:26:14.807495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:26:14.808385 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:26:14.808385 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:26:14.808385 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:26:14.808385 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:26:14.808385 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:26:14.808385 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:26:14.808385 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:26:14.808385 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:26:14.808385 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:26:14.808385 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:26:14.808385 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 19:26:14.808385 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:26:14.808385 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 553/1092] 10.222.0.15 () {56 vars in 1120 bytes} [Wed Jun 15 19:26:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:26:14.813668 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75072c8-5b8e-4e85-baaa-0fd16ef3c065 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:14.844449 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4403bdfd-5c84-4061-be52-fc9da21893f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:14.845903 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4403bdfd-5c84-4061-be52-fc9da21893f4 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:14.869443 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:14.912201 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75072c8-5b8e-4e85-baaa-0fd16ef3c065 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:14.943476 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 540/1093] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:26:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 19:26:14.946289 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6400bb-f2a9-45a2-8e75-d03fd5a08f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:14.947022 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6400bb-f2a9-45a2-8e75-d03fd5a08f89 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:14.947375 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6400bb-f2a9-45a2-8e75-d03fd5a08f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:14.947717 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6400bb-f2a9-45a2-8e75-d03fd5a08f89 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:14.948535 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6400bb-f2a9-45a2-8e75-d03fd5a08f89 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:14.948947 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6400bb-f2a9-45a2-8e75-d03fd5a08f89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:14.949822 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4403bdfd-5c84-4061-be52-fc9da21893f4 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:14.950231 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6400bb-f2a9-45a2-8e75-d03fd5a08f89 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:14.962784 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 554/1094] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Jun 15 19:26:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:26:14.964075 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6400bb-f2a9-45a2-8e75-d03fd5a08f89 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:14.985249 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:15.046833 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4403bdfd-5c84-4061-be52-fc9da21893f4 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:15.062142 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6400bb-f2a9-45a2-8e75-d03fd5a08f89 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:15.073504 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:15.085122 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f716ede-7f40-47f7-b45f-70ff38ddbb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:15.085972 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f716ede-7f40-47f7-b45f-70ff38ddbb42 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:15.086283 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f716ede-7f40-47f7-b45f-70ff38ddbb42 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:15.086611 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f716ede-7f40-47f7-b45f-70ff38ddbb42 None admin] PATH_INFO: `/v3/projects/c337126c82a1412e93a98ed265833fbd` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:15.087831 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f716ede-7f40-47f7-b45f-70ff38ddbb42 None admin] RBAC: Authorizing `identity:get_project(project_id=c337126c82a1412e93a98ed265833fbd)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:15.089261 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f716ede-7f40-47f7-b45f-70ff38ddbb42 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:15.090999 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 541/1095] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:26:14 2022] GET /identity/v3/projects/c337126c82a1412e93a98ed265833fbd => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:26:15.096835 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f716ede-7f40-47f7-b45f-70ff38ddbb42 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:15.100646 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d873e8-44c3-461f-9262-4651239fb2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:15.101960 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d873e8-44c3-461f-9262-4651239fb2d1 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:15.102275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d873e8-44c3-461f-9262-4651239fb2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:15.102554 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d873e8-44c3-461f-9262-4651239fb2d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:15.113430 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d873e8-44c3-461f-9262-4651239fb2d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:15.114900 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d873e8-44c3-461f-9262-4651239fb2d1 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:15.132675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-f3d873e8-44c3-461f-9262-4651239fb2d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:26:15.132675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:26:15.132675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:26:15.132675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:26:15.132675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:26:15.132675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:26:15.132675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:26:15.132675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:26:15.132675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:26:15.132675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:26:15.132675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:26:15.132675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:26:15.132675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:26:15.132675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:26:15.132675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:26:15.133674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:26:15.133674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:26:15.133674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:26:15.133674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:26:15.133674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:26:15.133674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:26:15.133674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:26:15.133674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:26:15.133674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:26:15.133674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:26:15.133674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:26:15.133674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:26:15.133674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:26:15.133674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:26:15.133674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:26:15.133674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:26:15.133674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:26:15.134637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:26:15.134637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:26:15.134637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:26:15.134637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:26:15.134637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 555/1096] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:26:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:26:15.139463 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d873e8-44c3-461f-9262-4651239fb2d1 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:15.193920 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f716ede-7f40-47f7-b45f-70ff38ddbb42 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:15.207221 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:15.228227 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a8cf6-423b-4d5b-845b-23fdd23a0ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:15.228877 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a8cf6-423b-4d5b-845b-23fdd23a0ce6 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:15.229160 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a8cf6-423b-4d5b-845b-23fdd23a0ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:15.229436 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a8cf6-423b-4d5b-845b-23fdd23a0ce6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:15.230300 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818a8cf6-423b-4d5b-845b-23fdd23a0ce6 None admin] RBAC: Adding query filter params (role.id=4f736313c26d4f8db48bbb2f346e50e4, scope.project.id=c337126c82a1412e93a98ed265833fbd, user.id=885a5bf7e411482a8071202cc12ea06b) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:15.230613 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818a8cf6-423b-4d5b-845b-23fdd23a0ce6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:15.232039 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818a8cf6-423b-4d5b-845b-23fdd23a0ce6 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:15.237739 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 542/1097] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:26:15 2022] GET /identity/v3/role_assignments?user.id=885a5bf7e411482a8071202cc12ea06b&scope.project.id=c337126c82a1412e93a98ed265833fbd&role.id=4f736313c26d4f8db48bbb2f346e50e4 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:26:15.240653 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d873e8-44c3-461f-9262-4651239fb2d1 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:15.277352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7097e34-ca0a-44d4-acbb-3e74fee3a052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:15.278030 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7097e34-ca0a-44d4-acbb-3e74fee3a052 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:15.278343 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7097e34-ca0a-44d4-acbb-3e74fee3a052 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:15.278618 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7097e34-ca0a-44d4-acbb-3e74fee3a052 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:15.279351 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7097e34-ca0a-44d4-acbb-3e74fee3a052 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:15.279659 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7097e34-ca0a-44d4-acbb-3e74fee3a052 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:15.281037 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7097e34-ca0a-44d4-acbb-3e74fee3a052 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:15.294786 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 556/1098] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:26:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:26:15.300665 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a8cf6-423b-4d5b-845b-23fdd23a0ce6 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:15.364489 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:15.411026 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a8cf6-423b-4d5b-845b-23fdd23a0ce6 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:15.446025 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8ee3c-5895-4c44-a205-f36cd80dd611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:15.446843 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8ee3c-5895-4c44-a205-f36cd80dd611 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:15.447125 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8ee3c-5895-4c44-a205-f36cd80dd611 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:15.447397 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8ee3c-5895-4c44-a205-f36cd80dd611 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:15.454976 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8ee3c-5895-4c44-a205-f36cd80dd611 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:15.456422 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8ee3c-5895-4c44-a205-f36cd80dd611 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:15.471577 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-b2e8ee3c-5895-4c44-a205-f36cd80dd611 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 19:26:15.471577 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:26:15.471577 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:26:15.471577 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:26:15.471577 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:26:15.471577 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:26:15.471577 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:26:15.471577 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:26:15.471577 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:26:15.471577 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:26:15.471577 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:26:15.471577 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:26:15.471577 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 19:26:15.471577 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 19:26:15.471577 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 19:26:15.471577 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 19:26:15.472585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:26:15.472585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:26:15.472585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:26:15.472585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:26:15.472585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:26:15.472585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:26:15.472585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:26:15.472585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:26:15.472585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:26:15.472585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:26:15.472585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:26:15.472585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:26:15.472585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:26:15.472585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:26:15.472585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:26:15.472585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:26:15.472585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:26:15.472585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:26:15.473860 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:26:15.473860 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:26:15.473860 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 19:26:15.473860 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 19:26:15.473860 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 19:26:15.473860 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 19:26:15.473860 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 19:26:15.473860 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 19:26:15.473860 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 19:26:15.473860 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:26:15.473860 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 543/1099] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Jun 15 19:26:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:26:15.479026 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7097e34-ca0a-44d4-acbb-3e74fee3a052 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:15.578399 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7097e34-ca0a-44d4-acbb-3e74fee3a052 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:15.607503 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:15.613242 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb55b4d3-3e30-46a2-85d3-db5c0b087dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:15.613908 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55b4d3-3e30-46a2-85d3-db5c0b087dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:15.614224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55b4d3-3e30-46a2-85d3-db5c0b087dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:15.614503 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55b4d3-3e30-46a2-85d3-db5c0b087dd9 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:15.615337 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb55b4d3-3e30-46a2-85d3-db5c0b087dd9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:15.615645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb55b4d3-3e30-46a2-85d3-db5c0b087dd9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:15.617599 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb55b4d3-3e30-46a2-85d3-db5c0b087dd9 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:15.634956 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 557/1100] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Jun 15 19:26:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:15.640566 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8ee3c-5895-4c44-a205-f36cd80dd611 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:15.737444 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8ee3c-5895-4c44-a205-f36cd80dd611 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:15.754039 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:15.772815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf01302-0e1c-42dd-aa82-34d92c2076cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:15.773462 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf01302-0e1c-42dd-aa82-34d92c2076cb None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:15.773746 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf01302-0e1c-42dd-aa82-34d92c2076cb None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:15.774020 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf01302-0e1c-42dd-aa82-34d92c2076cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:15.839484 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf01302-0e1c-42dd-aa82-34d92c2076cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:15.840924 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf01302-0e1c-42dd-aa82-34d92c2076cb None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:15.944074 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 544/1101] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:26:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 19:26:15.945178 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb55b4d3-3e30-46a2-85d3-db5c0b087dd9 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:16.041601 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb55b4d3-3e30-46a2-85d3-db5c0b087dd9 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:16.076147 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd2345c-509e-48a1-a926-7529c7b4a25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:16.076986 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2345c-509e-48a1-a926-7529c7b4a25f None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:16.077271 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2345c-509e-48a1-a926-7529c7b4a25f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:16.077549 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2345c-509e-48a1-a926-7529c7b4a25f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:16.087898 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd2345c-509e-48a1-a926-7529c7b4a25f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:16.089334 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd2345c-509e-48a1-a926-7529c7b4a25f None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:16.106904 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-efd2345c-509e-48a1-a926-7529c7b4a25f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:26:16.106904 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:26:16.106904 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:26:16.106904 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:26:16.106904 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:26:16.106904 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:26:16.106904 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:26:16.106904 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:26:16.106904 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:26:16.106904 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:26:16.106904 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:26:16.106904 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:26:16.106904 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:26:16.106904 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:26:16.106904 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:26:16.107979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:26:16.107979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:26:16.107979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:26:16.107979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:26:16.107979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:26:16.107979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:26:16.107979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:26:16.107979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:26:16.107979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:26:16.107979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:26:16.107979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:26:16.107979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:26:16.107979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:26:16.107979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:26:16.107979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:26:16.107979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:26:16.107979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:26:16.109099 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:26:16.109099 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:26:16.109099 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:26:16.109099 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:26:16.110230 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 558/1102] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:26:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:26:16.113686 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf01302-0e1c-42dd-aa82-34d92c2076cb None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:16.131183 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:16.210207 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf01302-0e1c-42dd-aa82-34d92c2076cb None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:16.243249 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700ed27-0693-4a89-a2f3-c67f944cb833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:16.243901 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700ed27-0693-4a89-a2f3-c67f944cb833 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:16.244195 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700ed27-0693-4a89-a2f3-c67f944cb833 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:16.244459 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700ed27-0693-4a89-a2f3-c67f944cb833 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:16.245169 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2700ed27-0693-4a89-a2f3-c67f944cb833 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:16.245473 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2700ed27-0693-4a89-a2f3-c67f944cb833 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:16.246864 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2700ed27-0693-4a89-a2f3-c67f944cb833 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:16.258014 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 545/1103] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:26:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:16.263762 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd2345c-509e-48a1-a926-7529c7b4a25f None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:16.306459 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:16.364911 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd2345c-509e-48a1-a926-7529c7b4a25f None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:16.398619 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e39cff-4afb-4344-aef2-364bc08b8a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:16.399469 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e39cff-4afb-4344-aef2-364bc08b8a5a None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:16.399751 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e39cff-4afb-4344-aef2-364bc08b8a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:16.400024 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e39cff-4afb-4344-aef2-364bc08b8a5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:16.406885 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e39cff-4afb-4344-aef2-364bc08b8a5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:16.408318 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e39cff-4afb-4344-aef2-364bc08b8a5a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:16.422443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-36e39cff-4afb-4344-aef2-364bc08b8a5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:26:16.422443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:26:16.422443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:26:16.422443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:26:16.422443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:26:16.422443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:26:16.422443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:26:16.422443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:26:16.422443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:26:16.422443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:26:16.422443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:26:16.422443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:26:16.422443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 19:26:16.422443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 19:26:16.422443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 19:26:16.423528 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 19:26:16.423528 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:26:16.423528 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:26:16.423528 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:26:16.423528 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:26:16.423528 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:26:16.423528 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:26:16.423528 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:26:16.423528 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:26:16.423528 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:26:16.423528 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:26:16.423528 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:26:16.423528 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:26:16.423528 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:26:16.423528 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:26:16.423528 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:26:16.423528 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:26:16.424545 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 19:26:16.424545 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 19:26:16.424545 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:26:16.424545 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:26:16.424545 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:26:16.424545 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:26:16.424545 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:26:16.424545 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:26:16.424545 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 559/1104] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:26:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:26:16.429379 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700ed27-0693-4a89-a2f3-c67f944cb833 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:16.505857 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:16.529708 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700ed27-0693-4a89-a2f3-c67f944cb833 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:16.570538 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9feebe0-30ec-4bfc-904d-664d4ee3173c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:16.571199 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9feebe0-30ec-4bfc-904d-664d4ee3173c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:16.571490 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9feebe0-30ec-4bfc-904d-664d4ee3173c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:16.571767 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9feebe0-30ec-4bfc-904d-664d4ee3173c None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:16.572555 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9feebe0-30ec-4bfc-904d-664d4ee3173c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:16.572866 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9feebe0-30ec-4bfc-904d-664d4ee3173c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:16.574316 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9feebe0-30ec-4bfc-904d-664d4ee3173c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:16.586003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 546/1105] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jun 15 19:26:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:16.591982 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e39cff-4afb-4344-aef2-364bc08b8a5a None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:16.690005 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e39cff-4afb-4344-aef2-364bc08b8a5a None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:16.691547 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:16.724637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c1c66-b0d7-4aaa-9f39-76947bb13c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:16.725278 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c1c66-b0d7-4aaa-9f39-76947bb13c70 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:16.725563 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c1c66-b0d7-4aaa-9f39-76947bb13c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:16.725840 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c1c66-b0d7-4aaa-9f39-76947bb13c70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:16.727181 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15c1c66-b0d7-4aaa-9f39-76947bb13c70 None admin] RBAC: Adding query filter params (role.id=34c882c657e2470e9bbe75d07694392d, scope.project.id=7fb8cc0eb560465e972ffd45867ee73b, user.id=3e74ac44e5814122a3226ac023f3688a) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:16.727496 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15c1c66-b0d7-4aaa-9f39-76947bb13c70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:16.728916 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15c1c66-b0d7-4aaa-9f39-76947bb13c70 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:16.734654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 560/1106] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:26:16 2022] GET /identity/v3/role_assignments?user.id=3e74ac44e5814122a3226ac023f3688a&scope.project.id=7fb8cc0eb560465e972ffd45867ee73b&role.id=34c882c657e2470e9bbe75d07694392d => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:17.375941 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2b9af-a237-4819-b730-6d1eda76addd None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:17.376442 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2b9af-a237-4819-b730-6d1eda76addd None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:17.376559 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2b9af-a237-4819-b730-6d1eda76addd None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:17.377877 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 547/1107] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:26:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:26:17.383831 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d376b-d0b5-47c9-abbb-d8a59331d023 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:17.384262 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d376b-d0b5-47c9-abbb-d8a59331d023 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:17.384387 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d376b-d0b5-47c9-abbb-d8a59331d023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:17.401888 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-8f0d376b-d0b5-47c9-abbb-d8a59331d023 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:26:17.403103 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-8f0d376b-d0b5-47c9-abbb-d8a59331d023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:26:17.472172 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 561/1108] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:26:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:26:17.496673 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc2b9af-a237-4819-b730-6d1eda76addd None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:17.598963 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc2b9af-a237-4819-b730-6d1eda76addd None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:17.631818 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fafd7e3-2726-4048-9d49-98adbede64ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:17.633145 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fafd7e3-2726-4048-9d49-98adbede64ab None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:17.633424 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fafd7e3-2726-4048-9d49-98adbede64ab None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:17.633695 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fafd7e3-2726-4048-9d49-98adbede64ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:17.634879 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fafd7e3-2726-4048-9d49-98adbede64ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:17.636319 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fafd7e3-2726-4048-9d49-98adbede64ab None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:17.637915 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 548/1109] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:26:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:17.643615 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d376b-d0b5-47c9-abbb-d8a59331d023 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:17.732892 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:17.738708 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d376b-d0b5-47c9-abbb-d8a59331d023 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:17.771496 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a0dfa0-786d-41a8-8613-5673d6448231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:17.772149 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a0dfa0-786d-41a8-8613-5673d6448231 None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:17.772431 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a0dfa0-786d-41a8-8613-5673d6448231 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:17.772729 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a0dfa0-786d-41a8-8613-5673d6448231 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:17.773434 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a0dfa0-786d-41a8-8613-5673d6448231 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:17.774873 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a0dfa0-786d-41a8-8613-5673d6448231 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:17.792900 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 562/1110] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Jun 15 19:26:17 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:26:19.934150 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf418515-29ce-4294-ad9c-cdc1026609db None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:19.934741 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf418515-29ce-4294-ad9c-cdc1026609db None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:19.934741 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf418515-29ce-4294-ad9c-cdc1026609db None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:19.936068 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 549/1111] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:26:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:26:19.941223 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6951e0-ea5f-442d-bcdb-a0b77f9d041e None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:19.941667 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6951e0-ea5f-442d-bcdb-a0b77f9d041e None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:19.941769 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6951e0-ea5f-442d-bcdb-a0b77f9d041e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:19.957996 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-fa6951e0-ea5f-442d-bcdb-a0b77f9d041e None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:26:19.959186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-fa6951e0-ea5f-442d-bcdb-a0b77f9d041e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:26:20.023597 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 563/1112] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:26:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 19:26:20.046699 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf418515-29ce-4294-ad9c-cdc1026609db None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:20.050466 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:20.146990 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf418515-29ce-4294-ad9c-cdc1026609db None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:20.179815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea64cf-ba10-4890-9c90-b75fc0201d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:20.180647 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea64cf-ba10-4890-9c90-b75fc0201d39 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:20.180927 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea64cf-ba10-4890-9c90-b75fc0201d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:20.181202 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea64cf-ba10-4890-9c90-b75fc0201d39 None admin] PATH_INFO: `/v3/roles/3920d5126e22480ab9a4eec9cf0fd4e7` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:20.182680 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fea64cf-ba10-4890-9c90-b75fc0201d39 None admin] RBAC: Authorizing `identity:get_role(role_id=3920d5126e22480ab9a4eec9cf0fd4e7)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:20.184036 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fea64cf-ba10-4890-9c90-b75fc0201d39 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:20.185423 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 550/1113] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:26:20 2022] GET /identity/v3/roles/3920d5126e22480ab9a4eec9cf0fd4e7 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:20.191069 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6951e0-ea5f-442d-bcdb-a0b77f9d041e None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:20.268095 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:20.286874 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6951e0-ea5f-442d-bcdb-a0b77f9d041e None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:20.319212 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4ae38-6538-45c5-81f0-0c78686b72f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:20.320112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4ae38-6538-45c5-81f0-0c78686b72f8 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:20.320396 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4ae38-6538-45c5-81f0-0c78686b72f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:20.320675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4ae38-6538-45c5-81f0-0c78686b72f8 None admin] PATH_INFO: `/v3/users/fe005d75a7df43edbd0a11e9d007ae70` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:20.333396 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a4ae38-6538-45c5-81f0-0c78686b72f8 None admin] RBAC: Authorizing `identity:get_user(user_id=fe005d75a7df43edbd0a11e9d007ae70)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:20.334856 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a4ae38-6538-45c5-81f0-0c78686b72f8 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:20.337081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 564/1114] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:26:20 2022] GET /identity/v3/users/fe005d75a7df43edbd0a11e9d007ae70 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:20.342904 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea64cf-ba10-4890-9c90-b75fc0201d39 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:20.411202 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:20.439066 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea64cf-ba10-4890-9c90-b75fc0201d39 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:20.472224 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e07bc-cc3e-40fb-abd2-038c2f4a9950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:20.472861 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e07bc-cc3e-40fb-abd2-038c2f4a9950 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:20.473142 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e07bc-cc3e-40fb-abd2-038c2f4a9950 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:20.473417 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e07bc-cc3e-40fb-abd2-038c2f4a9950 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:20.537695 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2e07bc-cc3e-40fb-abd2-038c2f4a9950 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:20.539193 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2e07bc-cc3e-40fb-abd2-038c2f4a9950 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:20.603603 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:20.632967 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 551/1115] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:26:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 19:26:20.638837 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4ae38-6538-45c5-81f0-0c78686b72f8 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:20.733732 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4ae38-6538-45c5-81f0-0c78686b72f8 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:20.767173 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5f668-92fd-4a5c-9a07-4bf35c021e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:20.768020 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5f668-92fd-4a5c-9a07-4bf35c021e10 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:20.768301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5f668-92fd-4a5c-9a07-4bf35c021e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:20.768576 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5f668-92fd-4a5c-9a07-4bf35c021e10 None admin] PATH_INFO: `/v3/projects/c337126c82a1412e93a98ed265833fbd` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:20.769734 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e5f668-92fd-4a5c-9a07-4bf35c021e10 None admin] RBAC: Authorizing `identity:get_project(project_id=c337126c82a1412e93a98ed265833fbd)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:20.771182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e5f668-92fd-4a5c-9a07-4bf35c021e10 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:20.772888 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 565/1116] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:26:20 2022] GET /identity/v3/projects/c337126c82a1412e93a98ed265833fbd => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:20.778899 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e07bc-cc3e-40fb-abd2-038c2f4a9950 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:20.878676 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e07bc-cc3e-40fb-abd2-038c2f4a9950 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:20.911254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b635f3-8a85-44f5-94e1-5f47ac0d15b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:20.911894 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b635f3-8a85-44f5-94e1-5f47ac0d15b1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:20.912177 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b635f3-8a85-44f5-94e1-5f47ac0d15b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:20.912454 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b635f3-8a85-44f5-94e1-5f47ac0d15b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:20.913308 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b635f3-8a85-44f5-94e1-5f47ac0d15b1 None admin] RBAC: Adding query filter params (role.id=3920d5126e22480ab9a4eec9cf0fd4e7, scope.project.id=c337126c82a1412e93a98ed265833fbd, user.id=fe005d75a7df43edbd0a11e9d007ae70) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:20.913622 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b635f3-8a85-44f5-94e1-5f47ac0d15b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:20.915066 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b635f3-8a85-44f5-94e1-5f47ac0d15b1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:20.920345 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 552/1117] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:26:20 2022] GET /identity/v3/role_assignments?user.id=fe005d75a7df43edbd0a11e9d007ae70&scope.project.id=c337126c82a1412e93a98ed265833fbd&role.id=3920d5126e22480ab9a4eec9cf0fd4e7 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:22.970477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963b7cd-6733-4bb0-ac02-328de4be6394 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:22.971028 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963b7cd-6733-4bb0-ac02-328de4be6394 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:22.971028 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963b7cd-6733-4bb0-ac02-328de4be6394 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:22.972397 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 566/1118] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:26:22.977541 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb86ecc-6554-492b-90d3-b5ecb0e8a5ff None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:22.977915 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb86ecc-6554-492b-90d3-b5ecb0e8a5ff None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:22.978106 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb86ecc-6554-492b-90d3-b5ecb0e8a5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:22.994877 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-3cb86ecc-6554-492b-90d3-b5ecb0e8a5ff None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:26:22.996054 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3cb86ecc-6554-492b-90d3-b5ecb0e8a5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:26:23.024029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:23.061130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 553/1119] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:26:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:26:23.083749 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963b7cd-6733-4bb0-ac02-328de4be6394 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:23.094354 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:23.181826 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963b7cd-6733-4bb0-ac02-328de4be6394 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:23.213699 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa5065-89fa-431c-b276-ab915a7331d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:23.214554 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa5065-89fa-431c-b276-ab915a7331d1 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:23.214836 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa5065-89fa-431c-b276-ab915a7331d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:23.215107 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa5065-89fa-431c-b276-ab915a7331d1 None admin] PATH_INFO: `/v3/roles/3920d5126e22480ab9a4eec9cf0fd4e7` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:23.216544 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa5065-89fa-431c-b276-ab915a7331d1 None admin] RBAC: Authorizing `identity:get_role(role_id=3920d5126e22480ab9a4eec9cf0fd4e7)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:23.217894 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa5065-89fa-431c-b276-ab915a7331d1 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:23.219265 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 567/1120] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:26:23 2022] GET /identity/v3/roles/3920d5126e22480ab9a4eec9cf0fd4e7 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:23.225008 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb86ecc-6554-492b-90d3-b5ecb0e8a5ff None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:23.320532 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb86ecc-6554-492b-90d3-b5ecb0e8a5ff None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:23.354213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba1f9e2-364a-4f02-960d-0748fa016ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:23.355081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1f9e2-364a-4f02-960d-0748fa016ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:23.355362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1f9e2-364a-4f02-960d-0748fa016ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:23.355638 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1f9e2-364a-4f02-960d-0748fa016ec8 None admin] PATH_INFO: `/v3/users/fe005d75a7df43edbd0a11e9d007ae70` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:23.356825 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba1f9e2-364a-4f02-960d-0748fa016ec8 None admin] RBAC: Authorizing `identity:get_user(user_id=fe005d75a7df43edbd0a11e9d007ae70)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:23.358293 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba1f9e2-364a-4f02-960d-0748fa016ec8 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:23.359924 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 554/1121] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:26:23 2022] GET /identity/v3/users/fe005d75a7df43edbd0a11e9d007ae70 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:23.365573 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa5065-89fa-431c-b276-ab915a7331d1 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:23.439253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:23.461370 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa5065-89fa-431c-b276-ab915a7331d1 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:23.494578 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7f2f55-5ee5-436a-aea8-2018196eceaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:23.495421 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f2f55-5ee5-436a-aea8-2018196eceaf None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:23.495700 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f2f55-5ee5-436a-aea8-2018196eceaf None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:23.495971 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f2f55-5ee5-436a-aea8-2018196eceaf None admin] PATH_INFO: `/v3/projects/c337126c82a1412e93a98ed265833fbd` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:23.497123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7f2f55-5ee5-436a-aea8-2018196eceaf None admin] RBAC: Authorizing `identity:get_project(project_id=c337126c82a1412e93a98ed265833fbd)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:23.498565 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7f2f55-5ee5-436a-aea8-2018196eceaf None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:23.500227 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 568/1122] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:26:23 2022] GET /identity/v3/projects/c337126c82a1412e93a98ed265833fbd => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:23.505994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba1f9e2-364a-4f02-960d-0748fa016ec8 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:23.539504 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:23.602978 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba1f9e2-364a-4f02-960d-0748fa016ec8 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:23.635910 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2a6c08-82b0-4adb-81ed-ff506ff1acc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:23.636778 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a6c08-82b0-4adb-81ed-ff506ff1acc9 None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:23.637060 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a6c08-82b0-4adb-81ed-ff506ff1acc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:23.637333 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a6c08-82b0-4adb-81ed-ff506ff1acc9 None admin] PATH_INFO: `/v3/projects/c337126c82a1412e93a98ed265833fbd/users/fe005d75a7df43edbd0a11e9d007ae70/roles/3920d5126e22480ab9a4eec9cf0fd4e7` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:23.639552 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2a6c08-82b0-4adb-81ed-ff506ff1acc9 None admin] RBAC: Authorizing `identity:create_grant(project_id=c337126c82a1412e93a98ed265833fbd, user_id=fe005d75a7df43edbd0a11e9d007ae70, role_id=3920d5126e22480ab9a4eec9cf0fd4e7)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:23.640971 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2a6c08-82b0-4adb-81ed-ff506ff1acc9 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:23.652900 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 555/1123] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jun 15 19:26:23 2022] PUT /identity/v3/projects/c337126c82a1412e93a98ed265833fbd/users/fe005d75a7df43edbd0a11e9d007ae70/roles/3920d5126e22480ab9a4eec9cf0fd4e7 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:26:25.724175 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df75db1-82a2-4fc9-a05a-631490775ad9 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:25.724676 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df75db1-82a2-4fc9-a05a-631490775ad9 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:25.724676 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df75db1-82a2-4fc9-a05a-631490775ad9 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:25.726005 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 569/1124] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:26:25.731127 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b3cdf-47fd-4c5e-988d-c75969a8e25f None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:25.731505 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b3cdf-47fd-4c5e-988d-c75969a8e25f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:25.731682 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b3cdf-47fd-4c5e-988d-c75969a8e25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:25.747807 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-2a1b3cdf-47fd-4c5e-988d-c75969a8e25f None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:26:25.748979 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-2a1b3cdf-47fd-4c5e-988d-c75969a8e25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:26:25.811941 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 556/1125] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:26:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:26:25.836355 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df75db1-82a2-4fc9-a05a-631490775ad9 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:25.941717 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df75db1-82a2-4fc9-a05a-631490775ad9 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:25.974783 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0798e9-6885-41c1-9810-4cb7e3349932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:25.975660 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0798e9-6885-41c1-9810-4cb7e3349932 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:25.975995 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0798e9-6885-41c1-9810-4cb7e3349932 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:25.976323 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0798e9-6885-41c1-9810-4cb7e3349932 None admin] PATH_INFO: `/v3/roles/3920d5126e22480ab9a4eec9cf0fd4e7` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:25.977757 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0798e9-6885-41c1-9810-4cb7e3349932 None admin] RBAC: Authorizing `identity:get_role(role_id=3920d5126e22480ab9a4eec9cf0fd4e7)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:25.979147 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0798e9-6885-41c1-9810-4cb7e3349932 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:25.980520 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 570/1126] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:26:25 2022] GET /identity/v3/roles/3920d5126e22480ab9a4eec9cf0fd4e7 => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:25.986349 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b3cdf-47fd-4c5e-988d-c75969a8e25f None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:26.011021 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:26.083568 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b3cdf-47fd-4c5e-988d-c75969a8e25f None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:26.115690 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79da3e2-5d73-452b-93ae-04a8633547f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:26.117053 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79da3e2-5d73-452b-93ae-04a8633547f8 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:26.117332 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79da3e2-5d73-452b-93ae-04a8633547f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:26.117606 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79da3e2-5d73-452b-93ae-04a8633547f8 None admin] PATH_INFO: `/v3/users/fe005d75a7df43edbd0a11e9d007ae70` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:26.118768 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79da3e2-5d73-452b-93ae-04a8633547f8 None admin] RBAC: Authorizing `identity:get_user(user_id=fe005d75a7df43edbd0a11e9d007ae70)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:26.120224 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79da3e2-5d73-452b-93ae-04a8633547f8 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:26.121827 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 557/1127] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:26:25 2022] GET /identity/v3/users/fe005d75a7df43edbd0a11e9d007ae70 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:26.127370 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0798e9-6885-41c1-9810-4cb7e3349932 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:26.143094 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:26.230135 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0798e9-6885-41c1-9810-4cb7e3349932 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:26.263255 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce87a6-e78d-45f8-8d28-3f1a23c90f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:26.263891 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce87a6-e78d-45f8-8d28-3f1a23c90f21 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:26.264176 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce87a6-e78d-45f8-8d28-3f1a23c90f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:26.264454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce87a6-e78d-45f8-8d28-3f1a23c90f21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:26.329837 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ce87a6-e78d-45f8-8d28-3f1a23c90f21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:26.331323 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ce87a6-e78d-45f8-8d28-3f1a23c90f21 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:26.344510 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:26.426346 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 571/1128] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:26:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 19:26:26.432884 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79da3e2-5d73-452b-93ae-04a8633547f8 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:26.523016 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:26.531779 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79da3e2-5d73-452b-93ae-04a8633547f8 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:26.565294 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8ed8d-ffc6-4a16-93a3-a9bb41126175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:26.566159 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8ed8d-ffc6-4a16-93a3-a9bb41126175 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:26.566449 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8ed8d-ffc6-4a16-93a3-a9bb41126175 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:26.566728 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8ed8d-ffc6-4a16-93a3-a9bb41126175 None admin] PATH_INFO: `/v3/projects/c337126c82a1412e93a98ed265833fbd` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:26.567905 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8ed8d-ffc6-4a16-93a3-a9bb41126175 None admin] RBAC: Authorizing `identity:get_project(project_id=c337126c82a1412e93a98ed265833fbd)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:26.569364 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8ed8d-ffc6-4a16-93a3-a9bb41126175 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:26.571111 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 558/1129] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:26:26 2022] GET /identity/v3/projects/c337126c82a1412e93a98ed265833fbd => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:26.577426 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce87a6-e78d-45f8-8d28-3f1a23c90f21 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:26.673820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce87a6-e78d-45f8-8d28-3f1a23c90f21 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:26.687197 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:26.708093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77a802-3b28-4c34-a5a0-1bf43160516f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:26.708729 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77a802-3b28-4c34-a5a0-1bf43160516f None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:26.709015 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77a802-3b28-4c34-a5a0-1bf43160516f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:26.709294 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77a802-3b28-4c34-a5a0-1bf43160516f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:26.710687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e77a802-3b28-4c34-a5a0-1bf43160516f None admin] RBAC: Adding query filter params (role.id=3920d5126e22480ab9a4eec9cf0fd4e7, scope.project.id=c337126c82a1412e93a98ed265833fbd, user.id=fe005d75a7df43edbd0a11e9d007ae70) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:26.711000 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e77a802-3b28-4c34-a5a0-1bf43160516f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:26.712402 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e77a802-3b28-4c34-a5a0-1bf43160516f None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:26.718329 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 572/1130] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:26:26 2022] GET /identity/v3/role_assignments?user.id=fe005d75a7df43edbd0a11e9d007ae70&scope.project.id=c337126c82a1412e93a98ed265833fbd&role.id=3920d5126e22480ab9a4eec9cf0fd4e7 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:28.826299 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82448e-504c-4c34-b8df-6b36f383f4a3 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:28.826822 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82448e-504c-4c34-b8df-6b36f383f4a3 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:28.827302 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82448e-504c-4c34-b8df-6b36f383f4a3 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:28.829249 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 559/1131] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:26:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:26:28.835525 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc8390-ecf2-4bd1-a53a-01782c6acfe4 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:28.835949 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc8390-ecf2-4bd1-a53a-01782c6acfe4 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:28.836088 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc8390-ecf2-4bd1-a53a-01782c6acfe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:28.857225 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-ebbc8390-ecf2-4bd1-a53a-01782c6acfe4 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:26:28.858445 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-ebbc8390-ecf2-4bd1-a53a-01782c6acfe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:26:28.927460 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 573/1132] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:26:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:26:28.950934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a82448e-504c-4c34-b8df-6b36f383f4a3 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:29.048611 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a82448e-504c-4c34-b8df-6b36f383f4a3 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:29.081410 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ef15ef-fc31-4dc5-8f77-74fd0f83e842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:29.082145 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ef15ef-fc31-4dc5-8f77-74fd0f83e842 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:29.082430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ef15ef-fc31-4dc5-8f77-74fd0f83e842 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:29.082711 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ef15ef-fc31-4dc5-8f77-74fd0f83e842 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:29.083885 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ef15ef-fc31-4dc5-8f77-74fd0f83e842 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:29.085852 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ef15ef-fc31-4dc5-8f77-74fd0f83e842 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:29.087454 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 560/1133] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:26:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:29.093295 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc8390-ecf2-4bd1-a53a-01782c6acfe4 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:29.189347 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc8390-ecf2-4bd1-a53a-01782c6acfe4 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:29.198404 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:29.225024 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aebeac-52ef-4a91-ac0f-5f28165b4683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:29.226283 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aebeac-52ef-4a91-ac0f-5f28165b4683 None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:29.226565 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aebeac-52ef-4a91-ac0f-5f28165b4683 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:29.226837 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aebeac-52ef-4a91-ac0f-5f28165b4683 None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:29.227532 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aebeac-52ef-4a91-ac0f-5f28165b4683 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:29.228946 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aebeac-52ef-4a91-ac0f-5f28165b4683 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:29.252521 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 574/1134] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Jun 15 19:26:29 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:26:31.352575 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03694b-f5a8-4654-a25e-bbaf1ff5ecc6 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:31.352981 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03694b-f5a8-4654-a25e-bbaf1ff5ecc6 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:31.353130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03694b-f5a8-4654-a25e-bbaf1ff5ecc6 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:31.354550 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 561/1135] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:26:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:26:31.359695 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b04e5-2e7b-4f8a-8e9c-61acf3c8ec3a None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:31.360077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b04e5-2e7b-4f8a-8e9c-61acf3c8ec3a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:31.360245 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b04e5-2e7b-4f8a-8e9c-61acf3c8ec3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:31.376874 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-1a7b04e5-2e7b-4f8a-8e9c-61acf3c8ec3a None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:26:31.378033 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-1a7b04e5-2e7b-4f8a-8e9c-61acf3c8ec3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:26:31.441548 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 575/1136] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:26:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:26:31.465798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03694b-f5a8-4654-a25e-bbaf1ff5ecc6 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:31.493282 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:31.564774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03694b-f5a8-4654-a25e-bbaf1ff5ecc6 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:31.597591 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c423822-db61-41f5-b6e6-c6d251adffa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:31.598331 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c423822-db61-41f5-b6e6-c6d251adffa9 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:31.598611 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c423822-db61-41f5-b6e6-c6d251adffa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:31.598882 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c423822-db61-41f5-b6e6-c6d251adffa9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:31.600032 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c423822-db61-41f5-b6e6-c6d251adffa9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:31.601936 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c423822-db61-41f5-b6e6-c6d251adffa9 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:31.603544 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 562/1137] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:26:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:31.609034 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b04e5-2e7b-4f8a-8e9c-61acf3c8ec3a None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:31.692971 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:31.705457 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b04e5-2e7b-4f8a-8e9c-61acf3c8ec3a None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:31.737583 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4697a4c7-4fe0-40a1-a654-5fffad22018a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:31.738268 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697a4c7-4fe0-40a1-a654-5fffad22018a None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:31.738549 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697a4c7-4fe0-40a1-a654-5fffad22018a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:31.738821 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697a4c7-4fe0-40a1-a654-5fffad22018a None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:31.739525 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4697a4c7-4fe0-40a1-a654-5fffad22018a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:31.741479 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4697a4c7-4fe0-40a1-a654-5fffad22018a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:31.760058 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 576/1138] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Jun 15 19:26:31 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:26:33.883165 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc68a5-187e-4cec-bfff-fbff46db855e None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:33.883689 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc68a5-187e-4cec-bfff-fbff46db855e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:33.883780 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc68a5-187e-4cec-bfff-fbff46db855e None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:33.885083 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 563/1139] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:26:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:26:33.890315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20427493-6fda-4f85-9431-0252b6716a20 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:33.890744 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20427493-6fda-4f85-9431-0252b6716a20 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:33.890881 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20427493-6fda-4f85-9431-0252b6716a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:33.907766 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-20427493-6fda-4f85-9431-0252b6716a20 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:26:33.908935 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-20427493-6fda-4f85-9431-0252b6716a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:26:33.973407 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 577/1140] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:26:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:26:33.996619 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc68a5-187e-4cec-bfff-fbff46db855e None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:34.089697 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:34.096511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc68a5-187e-4cec-bfff-fbff46db855e None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:34.128765 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc55853-184c-486d-b41a-185808b80b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:34.129537 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc55853-184c-486d-b41a-185808b80b0b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:34.129822 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc55853-184c-486d-b41a-185808b80b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:34.130114 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc55853-184c-486d-b41a-185808b80b0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:34.143256 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc55853-184c-486d-b41a-185808b80b0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:26:34.152016 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-2cc55853-184c-486d-b41a-185808b80b0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:26:34.152016 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:26:34.152016 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:26:34.152016 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:26:34.152016 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:26:34.152016 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:26:34.152016 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:26:34.152016 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:26:34.152016 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:26:34.152016 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:26:34.152016 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:26:34.152016 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:26:34.152016 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:26:34.152016 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:26:34.152016 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:26:34.152016 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:26:34.153248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:26:34.153248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:26:34.153248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:26:34.153248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:26:34.153248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:26:34.153248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:26:34.153248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:26:34.153248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:26:34.153248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:26:34.153248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:26:34.153248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:26:34.153248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:26:34.153248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:26:34.153248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:26:34.153248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:26:34.153248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:26:34.154213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:26:34.154213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:26:34.154213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:26:34.154213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:26:34.154213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:26:34.154213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:26:34.154213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:26:34.154213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:26:34.154213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:26:34.154213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:26:34.154213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:26:34.154213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:26:34.154213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 564/1141] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jun 15 19:26:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:26:34.159315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20427493-6fda-4f85-9431-0252b6716a20 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:34.229768 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:34.256035 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20427493-6fda-4f85-9431-0252b6716a20 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:34.288477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec48c5a-3848-48c9-b0d8-5fb70f207e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:34.289130 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec48c5a-3848-48c9-b0d8-5fb70f207e37 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:34.289519 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec48c5a-3848-48c9-b0d8-5fb70f207e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:34.289796 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec48c5a-3848-48c9-b0d8-5fb70f207e37 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:34.290575 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec48c5a-3848-48c9-b0d8-5fb70f207e37 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:34.290883 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec48c5a-3848-48c9-b0d8-5fb70f207e37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:34.292245 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec48c5a-3848-48c9-b0d8-5fb70f207e37 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:34.300734 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 578/1142] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:26:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:34.307106 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc55853-184c-486d-b41a-185808b80b0b None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:34.407389 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc55853-184c-486d-b41a-185808b80b0b None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:34.439261 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc99b2-69cb-44aa-b42f-bdf855043b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:34.440109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc99b2-69cb-44aa-b42f-bdf855043b19 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:34.440390 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc99b2-69cb-44aa-b42f-bdf855043b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:34.440662 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc99b2-69cb-44aa-b42f-bdf855043b19 None admin] PATH_INFO: `/v3/users/93a3e8e624f64122bcae5c3a8497d55c` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:34.453666 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cc99b2-69cb-44aa-b42f-bdf855043b19 None admin] RBAC: Authorizing `identity:get_user(user_id=93a3e8e624f64122bcae5c3a8497d55c)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:34.455119 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cc99b2-69cb-44aa-b42f-bdf855043b19 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:34.456725 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 565/1143] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:26:34 2022] GET /identity/v3/users/93a3e8e624f64122bcae5c3a8497d55c => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:34.462441 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec48c5a-3848-48c9-b0d8-5fb70f207e37 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:34.557010 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec48c5a-3848-48c9-b0d8-5fb70f207e37 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:34.588913 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77899277-6dcf-4f0a-8c26-f75d21104b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:34.589535 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77899277-6dcf-4f0a-8c26-f75d21104b91 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:34.589816 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77899277-6dcf-4f0a-8c26-f75d21104b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:34.590110 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77899277-6dcf-4f0a-8c26-f75d21104b91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:34.593379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:34.653924 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77899277-6dcf-4f0a-8c26-f75d21104b91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:34.655388 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77899277-6dcf-4f0a-8c26-f75d21104b91 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:34.748202 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 579/1144] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:26:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 19:26:34.754010 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc99b2-69cb-44aa-b42f-bdf855043b19 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:34.769960 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:34.852072 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc99b2-69cb-44aa-b42f-bdf855043b19 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:34.885094 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35177d3d-9119-438c-91b6-7f90667efcc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:34.885937 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35177d3d-9119-438c-91b6-7f90667efcc3 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:34.886251 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35177d3d-9119-438c-91b6-7f90667efcc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:34.886530 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35177d3d-9119-438c-91b6-7f90667efcc3 None admin] PATH_INFO: `/v3/projects/428e113bbc0e4d5597f45ae3e8ca31a8` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:34.887707 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35177d3d-9119-438c-91b6-7f90667efcc3 None admin] RBAC: Authorizing `identity:get_project(project_id=428e113bbc0e4d5597f45ae3e8ca31a8)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:34.889142 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35177d3d-9119-438c-91b6-7f90667efcc3 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:34.890847 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 566/1145] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:26:34 2022] GET /identity/v3/projects/428e113bbc0e4d5597f45ae3e8ca31a8 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:34.896866 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77899277-6dcf-4f0a-8c26-f75d21104b91 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:34.930021 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:34.992858 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77899277-6dcf-4f0a-8c26-f75d21104b91 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:35.025565 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de7585-4069-43f5-b7a4-c81895e599eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:35.026174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de7585-4069-43f5-b7a4-c81895e599eb None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:35.026460 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de7585-4069-43f5-b7a4-c81895e599eb None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:35.026736 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de7585-4069-43f5-b7a4-c81895e599eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:35.027556 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de7585-4069-43f5-b7a4-c81895e599eb None admin] RBAC: Adding query filter params (role.id=4f736313c26d4f8db48bbb2f346e50e4, scope.project.id=428e113bbc0e4d5597f45ae3e8ca31a8, user.id=93a3e8e624f64122bcae5c3a8497d55c) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:35.027866 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de7585-4069-43f5-b7a4-c81895e599eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:35.029276 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de7585-4069-43f5-b7a4-c81895e599eb None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:35.034578 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 580/1146] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:26:34 2022] GET /identity/v3/role_assignments?user.id=93a3e8e624f64122bcae5c3a8497d55c&scope.project.id=428e113bbc0e4d5597f45ae3e8ca31a8&role.id=4f736313c26d4f8db48bbb2f346e50e4 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:37.095345 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b96b8-df0b-49da-8d3d-a4ad4225df6b None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:37.095904 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b96b8-df0b-49da-8d3d-a4ad4225df6b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:37.095904 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b96b8-df0b-49da-8d3d-a4ad4225df6b None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:37.097254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 567/1147] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:26:37.102866 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bd305-1e10-4cd2-8365-3346fb3f822b None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:37.103284 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bd305-1e10-4cd2-8365-3346fb3f822b None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:37.103458 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bd305-1e10-4cd2-8365-3346fb3f822b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:37.120338 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-192bd305-1e10-4cd2-8365-3346fb3f822b None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:26:37.121514 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-192bd305-1e10-4cd2-8365-3346fb3f822b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:26:37.184871 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 581/1148] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:26:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:26:37.207109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29b96b8-df0b-49da-8d3d-a4ad4225df6b None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:37.286334 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:37.305137 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29b96b8-df0b-49da-8d3d-a4ad4225df6b None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:37.337614 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6b6cf-5928-4e84-aff3-3b2ec2966d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:37.338418 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6b6cf-5928-4e84-aff3-3b2ec2966d2a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:37.338707 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6b6cf-5928-4e84-aff3-3b2ec2966d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:37.338976 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6b6cf-5928-4e84-aff3-3b2ec2966d2a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:37.349069 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd6b6cf-5928-4e84-aff3-3b2ec2966d2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:26:37.357186 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-8cd6b6cf-5928-4e84-aff3-3b2ec2966d2a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:26:37.357186 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:26:37.357186 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:26:37.357186 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:26:37.357186 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:26:37.357186 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:26:37.357186 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:26:37.357186 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:26:37.357186 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:26:37.357186 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:26:37.357186 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:26:37.357186 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:26:37.357186 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:26:37.357186 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:26:37.357186 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:26:37.357186 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:26:37.358136 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:26:37.358136 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:26:37.358136 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:26:37.358136 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:26:37.358136 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:26:37.358136 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:26:37.358136 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:26:37.358136 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:26:37.358136 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:26:37.358136 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:26:37.358136 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:26:37.358136 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:26:37.358136 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:26:37.358136 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:26:37.358136 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:26:37.358136 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:26:37.358999 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:26:37.358999 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:26:37.358999 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:26:37.358999 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:26:37.358999 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:26:37.358999 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:26:37.358999 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:26:37.358999 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:26:37.358999 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:26:37.358999 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:26:37.358999 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:26:37.358999 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:26:37.358999 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 568/1149] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jun 15 19:26:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:26:37.364440 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bd305-1e10-4cd2-8365-3346fb3f822b None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:37.393013 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:37.461933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bd305-1e10-4cd2-8365-3346fb3f822b None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:37.494852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b350f2-747a-4f08-be93-20cad577065b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:37.495499 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b350f2-747a-4f08-be93-20cad577065b None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:37.495778 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b350f2-747a-4f08-be93-20cad577065b None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:37.496051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b350f2-747a-4f08-be93-20cad577065b None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:37.496781 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b350f2-747a-4f08-be93-20cad577065b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:37.497082 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b350f2-747a-4f08-be93-20cad577065b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:37.498456 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b350f2-747a-4f08-be93-20cad577065b None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:37.506128 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 582/1150] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:26:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:37.512079 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6b6cf-5928-4e84-aff3-3b2ec2966d2a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:37.607977 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6b6cf-5928-4e84-aff3-3b2ec2966d2a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:37.640206 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adccdb7-e2af-4a86-982c-a295378f8ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:37.641051 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adccdb7-e2af-4a86-982c-a295378f8ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:37.641335 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adccdb7-e2af-4a86-982c-a295378f8ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:37.641612 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adccdb7-e2af-4a86-982c-a295378f8ed9 None admin] PATH_INFO: `/v3/users/93a3e8e624f64122bcae5c3a8497d55c` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:37.642766 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adccdb7-e2af-4a86-982c-a295378f8ed9 None admin] RBAC: Authorizing `identity:get_user(user_id=93a3e8e624f64122bcae5c3a8497d55c)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:37.644223 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adccdb7-e2af-4a86-982c-a295378f8ed9 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:37.645808 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 569/1151] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:26:37 2022] GET /identity/v3/users/93a3e8e624f64122bcae5c3a8497d55c => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:37.651453 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b350f2-747a-4f08-be93-20cad577065b None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:37.738053 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:37.747444 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b350f2-747a-4f08-be93-20cad577065b None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:37.779028 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b96299-e92f-44a1-b6b8-f450060e4e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:37.779866 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b96299-e92f-44a1-b6b8-f450060e4e14 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:37.780143 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b96299-e92f-44a1-b6b8-f450060e4e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:37.780413 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b96299-e92f-44a1-b6b8-f450060e4e14 None admin] PATH_INFO: `/v3/projects/428e113bbc0e4d5597f45ae3e8ca31a8` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:37.781562 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b96299-e92f-44a1-b6b8-f450060e4e14 None admin] RBAC: Authorizing `identity:get_project(project_id=428e113bbc0e4d5597f45ae3e8ca31a8)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:37.782996 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b96299-e92f-44a1-b6b8-f450060e4e14 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:37.784666 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 583/1152] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:26:37 2022] GET /identity/v3/projects/428e113bbc0e4d5597f45ae3e8ca31a8 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:37.791018 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adccdb7-e2af-4a86-982c-a295378f8ed9 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:37.804705 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:37.890002 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adccdb7-e2af-4a86-982c-a295378f8ed9 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:37.921803 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a13cc3-2dc1-4669-bbbb-84d139514ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:37.922697 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a13cc3-2dc1-4669-bbbb-84d139514ca3 None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:37.922977 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a13cc3-2dc1-4669-bbbb-84d139514ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:37.923248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a13cc3-2dc1-4669-bbbb-84d139514ca3 None admin] PATH_INFO: `/v3/projects/428e113bbc0e4d5597f45ae3e8ca31a8/users/93a3e8e624f64122bcae5c3a8497d55c/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:37.925171 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a13cc3-2dc1-4669-bbbb-84d139514ca3 None admin] RBAC: Authorizing `identity:create_grant(project_id=428e113bbc0e4d5597f45ae3e8ca31a8, user_id=93a3e8e624f64122bcae5c3a8497d55c, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:37.926722 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a13cc3-2dc1-4669-bbbb-84d139514ca3 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:37.940151 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 570/1153] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jun 15 19:26:37 2022] PUT /identity/v3/projects/428e113bbc0e4d5597f45ae3e8ca31a8/users/93a3e8e624f64122bcae5c3a8497d55c/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:26:40.011231 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ec524-d609-4446-a458-89918ee3fddc None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:40.011788 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ec524-d609-4446-a458-89918ee3fddc None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:40.011788 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ec524-d609-4446-a458-89918ee3fddc None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:40.013153 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 584/1154] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:26:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:26:40.018231 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b501e3f-c61f-4275-b21e-bd1f6b379da1 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:40.018620 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b501e3f-c61f-4275-b21e-bd1f6b379da1 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:40.018790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b501e3f-c61f-4275-b21e-bd1f6b379da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:40.037017 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-6b501e3f-c61f-4275-b21e-bd1f6b379da1 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:26:40.038764 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6b501e3f-c61f-4275-b21e-bd1f6b379da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:26:40.098441 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:40.104205 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 571/1155] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:26:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:26:40.127099 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ec524-d609-4446-a458-89918ee3fddc None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:40.224658 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ec524-d609-4446-a458-89918ee3fddc None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:40.257085 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80cc366-b29d-4fe3-ad4b-efa2ffe33738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:40.257861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80cc366-b29d-4fe3-ad4b-efa2ffe33738 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:40.258167 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80cc366-b29d-4fe3-ad4b-efa2ffe33738 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:40.258443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80cc366-b29d-4fe3-ad4b-efa2ffe33738 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:40.270421 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80cc366-b29d-4fe3-ad4b-efa2ffe33738 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:26:40.278524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-e80cc366-b29d-4fe3-ad4b-efa2ffe33738 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:26:40.278524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:26:40.278524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:26:40.278524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:26:40.278524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:26:40.278524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:26:40.278524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:26:40.278524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:26:40.278524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:26:40.278524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:26:40.278524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:26:40.278524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:26:40.278524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:26:40.278524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:26:40.278524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:26:40.278524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:26:40.279470 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:26:40.279470 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:26:40.279470 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:26:40.279470 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:26:40.279470 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:26:40.279470 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:26:40.279470 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:26:40.279470 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:26:40.279470 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:26:40.279470 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:26:40.279470 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:26:40.279470 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:26:40.279470 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:26:40.279470 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:26:40.279470 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:26:40.279470 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:26:40.280315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:26:40.280315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:26:40.280315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:26:40.280315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:26:40.280315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:26:40.280315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:26:40.280315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:26:40.280315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:26:40.280315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:26:40.280315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:26:40.280315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:26:40.280315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:26:40.280315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 585/1156] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jun 15 19:26:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:26:40.285647 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b501e3f-c61f-4275-b21e-bd1f6b379da1 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:40.381246 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b501e3f-c61f-4275-b21e-bd1f6b379da1 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:40.414496 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a040496-16d7-47f9-b4c2-979f0224b881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:40.415152 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a040496-16d7-47f9-b4c2-979f0224b881 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:40.415435 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a040496-16d7-47f9-b4c2-979f0224b881 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:40.415708 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a040496-16d7-47f9-b4c2-979f0224b881 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:40.416450 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a040496-16d7-47f9-b4c2-979f0224b881 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:40.416758 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a040496-16d7-47f9-b4c2-979f0224b881 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:40.418137 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a040496-16d7-47f9-b4c2-979f0224b881 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:40.425865 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 572/1157] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:26:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:40.431742 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80cc366-b29d-4fe3-ad4b-efa2ffe33738 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:40.453233 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:40.528440 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80cc366-b29d-4fe3-ad4b-efa2ffe33738 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:40.561618 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91daa4ab-83b7-4ca2-89b7-ba737e86b051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:40.562512 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91daa4ab-83b7-4ca2-89b7-ba737e86b051 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:40.562795 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91daa4ab-83b7-4ca2-89b7-ba737e86b051 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:40.563072 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91daa4ab-83b7-4ca2-89b7-ba737e86b051 None admin] PATH_INFO: `/v3/users/93a3e8e624f64122bcae5c3a8497d55c` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:40.564265 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91daa4ab-83b7-4ca2-89b7-ba737e86b051 None admin] RBAC: Authorizing `identity:get_user(user_id=93a3e8e624f64122bcae5c3a8497d55c)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:40.565695 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91daa4ab-83b7-4ca2-89b7-ba737e86b051 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:40.567382 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 586/1158] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:26:40 2022] GET /identity/v3/users/93a3e8e624f64122bcae5c3a8497d55c => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:40.573133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a040496-16d7-47f9-b4c2-979f0224b881 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:40.633433 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:40.670960 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a040496-16d7-47f9-b4c2-979f0224b881 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:40.702727 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea14ea1-f281-40e2-9b88-93f4528f3b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:40.703354 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea14ea1-f281-40e2-9b88-93f4528f3b12 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:40.703634 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea14ea1-f281-40e2-9b88-93f4528f3b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:40.703906 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea14ea1-f281-40e2-9b88-93f4528f3b12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:40.767701 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea14ea1-f281-40e2-9b88-93f4528f3b12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:40.769132 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea14ea1-f281-40e2-9b88-93f4528f3b12 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:40.826527 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:40.864704 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 573/1159] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:26:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 19:26:40.871191 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91daa4ab-83b7-4ca2-89b7-ba737e86b051 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:40.955592 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:40.967810 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91daa4ab-83b7-4ca2-89b7-ba737e86b051 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:40.999751 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fddf3b-4ec4-411e-a705-76a993d45640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:41.000596 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fddf3b-4ec4-411e-a705-76a993d45640 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:41.000879 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fddf3b-4ec4-411e-a705-76a993d45640 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:41.001154 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fddf3b-4ec4-411e-a705-76a993d45640 None admin] PATH_INFO: `/v3/projects/428e113bbc0e4d5597f45ae3e8ca31a8` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:41.002352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fddf3b-4ec4-411e-a705-76a993d45640 None admin] RBAC: Authorizing `identity:get_project(project_id=428e113bbc0e4d5597f45ae3e8ca31a8)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:41.003775 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fddf3b-4ec4-411e-a705-76a993d45640 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:41.005463 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 587/1160] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:26:40 2022] GET /identity/v3/projects/428e113bbc0e4d5597f45ae3e8ca31a8 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:41.011749 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea14ea1-f281-40e2-9b88-93f4528f3b12 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:41.106628 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea14ea1-f281-40e2-9b88-93f4528f3b12 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:41.139074 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9daaaa-ec5c-4058-b7a0-3919ea1bda2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:41.139704 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9daaaa-ec5c-4058-b7a0-3919ea1bda2f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:41.139983 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9daaaa-ec5c-4058-b7a0-3919ea1bda2f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:41.140252 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9daaaa-ec5c-4058-b7a0-3919ea1bda2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:41.141071 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9daaaa-ec5c-4058-b7a0-3919ea1bda2f None admin] RBAC: Adding query filter params (role.id=4f736313c26d4f8db48bbb2f346e50e4, scope.project.id=428e113bbc0e4d5597f45ae3e8ca31a8, user.id=93a3e8e624f64122bcae5c3a8497d55c) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:41.141381 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9daaaa-ec5c-4058-b7a0-3919ea1bda2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:41.142816 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9daaaa-ec5c-4058-b7a0-3919ea1bda2f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:41.148880 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 574/1161] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:26:41 2022] GET /identity/v3/role_assignments?user.id=93a3e8e624f64122bcae5c3a8497d55c&scope.project.id=428e113bbc0e4d5597f45ae3e8ca31a8&role.id=4f736313c26d4f8db48bbb2f346e50e4 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:43.254766 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b74b2-c273-4415-8f77-b16583af0213 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:43.255292 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b74b2-c273-4415-8f77-b16583af0213 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:43.255292 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b74b2-c273-4415-8f77-b16583af0213 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:43.257012 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 588/1162] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:26:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:26:43.262185 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd069cb-bcee-4256-9055-794ae4dec6d4 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:43.262582 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd069cb-bcee-4256-9055-794ae4dec6d4 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:43.262744 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd069cb-bcee-4256-9055-794ae4dec6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:43.279494 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-3cd069cb-bcee-4256-9055-794ae4dec6d4 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:26:43.280670 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3cd069cb-bcee-4256-9055-794ae4dec6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:26:43.300588 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:43.347174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 575/1163] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:26:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:26:43.369194 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b74b2-c273-4415-8f77-b16583af0213 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:43.466618 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b74b2-c273-4415-8f77-b16583af0213 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:43.499517 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeea04a-69cb-4209-9dcc-2221fb00a410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:43.500144 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeea04a-69cb-4209-9dcc-2221fb00a410 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:43.500430 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeea04a-69cb-4209-9dcc-2221fb00a410 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:43.500705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeea04a-69cb-4209-9dcc-2221fb00a410 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:43.524577 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:43.565371 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeea04a-69cb-4209-9dcc-2221fb00a410 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:43.566828 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeea04a-69cb-4209-9dcc-2221fb00a410 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:43.659536 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 589/1164] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:26:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 19:26:43.665808 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd069cb-bcee-4256-9055-794ae4dec6d4 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:43.760154 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd069cb-bcee-4256-9055-794ae4dec6d4 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:43.792338 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a287ed33-6f7b-476c-8f30-1a63b08d01d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:43.793054 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287ed33-6f7b-476c-8f30-1a63b08d01d1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:43.793333 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287ed33-6f7b-476c-8f30-1a63b08d01d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:43.793603 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287ed33-6f7b-476c-8f30-1a63b08d01d1 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:43.800026 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a287ed33-6f7b-476c-8f30-1a63b08d01d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:43.801461 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a287ed33-6f7b-476c-8f30-1a63b08d01d1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:43.815658 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-a287ed33-6f7b-476c-8f30-1a63b08d01d1 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 15 19:26:43.815658 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:26:43.815658 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 15 19:26:43.815658 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 15 19:26:43.815658 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:26:43.815658 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:26:43.815658 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:26:43.815658 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:26:43.815658 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 15 19:26:43.815658 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:26:43.815658 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 19:26:43.815658 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:26:43.815658 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:26:43.815658 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:26:43.815658 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:26:43.815658 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:26:43.815658 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:26:43.816885 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:26:43.816885 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:26:43.816885 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:26:43.816885 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:26:43.816885 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:26:43.816885 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:26:43.816885 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:26:43.816885 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:26:43.816885 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:26:43.816885 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:26:43.816885 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:26:43.816885 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:26:43.816885 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:26:43.816885 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:26:43.816885 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:26:43.816885 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:26:43.817915 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:26:43.817915 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:26:43.817915 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:26:43.817915 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:26:43.817915 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:26:43.817915 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:26:43.817915 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:26:43.817915 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:26:43.817915 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:26:43.817915 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:26:43.817915 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 15 19:26:43.817915 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:26:43.817915 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 15 19:26:43.817915 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:26:43.817915 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 576/1165] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Jun 15 19:26:43 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:26:43.822737 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeea04a-69cb-4209-9dcc-2221fb00a410 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:43.879275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:43.921756 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeea04a-69cb-4209-9dcc-2221fb00a410 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:43.954201 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b82ac46-d57c-4f8a-810c-f4424a6290ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:43.954856 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82ac46-d57c-4f8a-810c-f4424a6290ce None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:43.955139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82ac46-d57c-4f8a-810c-f4424a6290ce None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:43.955415 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82ac46-d57c-4f8a-810c-f4424a6290ce None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:43.956128 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b82ac46-d57c-4f8a-810c-f4424a6290ce None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:43.956466 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b82ac46-d57c-4f8a-810c-f4424a6290ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:43.957819 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b82ac46-d57c-4f8a-810c-f4424a6290ce None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:43.965334 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 590/1166] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Jun 15 19:26:43 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:46.033519 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32e94a-6761-40e2-bf35-cc92c76c71fc None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:46.034117 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32e94a-6761-40e2-bf35-cc92c76c71fc None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:46.034117 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32e94a-6761-40e2-bf35-cc92c76c71fc None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:46.035448 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 577/1167] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:26:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:26:46.041053 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12164a3e-3d2b-4926-ab45-4a5cb2f65c7d None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:46.041433 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12164a3e-3d2b-4926-ab45-4a5cb2f65c7d None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:46.041614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12164a3e-3d2b-4926-ab45-4a5cb2f65c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:46.058617 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-12164a3e-3d2b-4926-ab45-4a5cb2f65c7d None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:26:46.059809 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-12164a3e-3d2b-4926-ab45-4a5cb2f65c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:26:46.123390 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 591/1168] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:26:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:26:46.150162 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca32e94a-6761-40e2-bf35-cc92c76c71fc None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:46.170981 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:46.251751 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca32e94a-6761-40e2-bf35-cc92c76c71fc None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:46.284060 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96aec2-dd6b-41c4-aa7d-44b634528711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:46.284651 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96aec2-dd6b-41c4-aa7d-44b634528711 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:46.284933 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96aec2-dd6b-41c4-aa7d-44b634528711 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:46.285206 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96aec2-dd6b-41c4-aa7d-44b634528711 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:46.285933 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf96aec2-dd6b-41c4-aa7d-44b634528711 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:46.287947 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf96aec2-dd6b-41c4-aa7d-44b634528711 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:46.301750 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 578/1169] 10.222.0.15 () {60 vars in 1138 bytes} [Wed Jun 15 19:26:46 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:26:48.407422 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea783d5-4f09-472d-bcd8-9831484fbb4a None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:48.407929 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea783d5-4f09-472d-bcd8-9831484fbb4a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:48.408046 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea783d5-4f09-472d-bcd8-9831484fbb4a None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:48.409340 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 592/1170] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:26:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:26:48.414479 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83bf203-2f3e-4b17-9870-12634415c117 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:48.414916 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83bf203-2f3e-4b17-9870-12634415c117 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:48.415036 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83bf203-2f3e-4b17-9870-12634415c117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:48.431444 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-f83bf203-2f3e-4b17-9870-12634415c117 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:26:48.432623 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f83bf203-2f3e-4b17-9870-12634415c117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:26:48.497271 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 579/1171] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:26:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:26:48.520632 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea783d5-4f09-472d-bcd8-9831484fbb4a None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:48.571008 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:48.619335 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea783d5-4f09-472d-bcd8-9831484fbb4a None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:48.651872 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4980c0-bbcb-4f46-8d36-b99b81efc6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:48.652714 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4980c0-bbcb-4f46-8d36-b99b81efc6a8 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:48.652993 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4980c0-bbcb-4f46-8d36-b99b81efc6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:48.653262 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4980c0-bbcb-4f46-8d36-b99b81efc6a8 None admin] PATH_INFO: `/v3/domains/b5ab6e94c4a34e8f8ca7f015b4d4a23c` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:48.663217 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4980c0-bbcb-4f46-8d36-b99b81efc6a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=b5ab6e94c4a34e8f8ca7f015b4d4a23c)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:48.664635 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4980c0-bbcb-4f46-8d36-b99b81efc6a8 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:48.666265 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 593/1172] 10.222.0.15 () {56 vars in 1183 bytes} [Wed Jun 15 19:26:48 2022] GET /identity/v3/domains/b5ab6e94c4a34e8f8ca7f015b4d4a23c => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:48.672070 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83bf203-2f3e-4b17-9870-12634415c117 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:48.680460 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:48.769397 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83bf203-2f3e-4b17-9870-12634415c117 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:48.801435 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af41f47-6fb9-4c00-99de-213b288a113b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:48.802079 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af41f47-6fb9-4c00-99de-213b288a113b None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:48.802914 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af41f47-6fb9-4c00-99de-213b288a113b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:48.803190 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af41f47-6fb9-4c00-99de-213b288a113b None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:48.803890 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af41f47-6fb9-4c00-99de-213b288a113b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:48.805319 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af41f47-6fb9-4c00-99de-213b288a113b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:48.831695 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 580/1173] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Jun 15 19:26:48 2022] POST /identity/v3/projects => generated 361 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:26:50.946882 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced278a-abb4-4908-9ae5-58789093ec29 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:50.947473 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced278a-abb4-4908-9ae5-58789093ec29 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:50.947473 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced278a-abb4-4908-9ae5-58789093ec29 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:50.948801 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 594/1174] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:26:50.954217 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2526cd1-77d7-4d01-b399-dc044487526a None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:50.954663 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2526cd1-77d7-4d01-b399-dc044487526a None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:50.954766 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2526cd1-77d7-4d01-b399-dc044487526a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:50.971049 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-e2526cd1-77d7-4d01-b399-dc044487526a None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:26:50.972216 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e2526cd1-77d7-4d01-b399-dc044487526a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:26:51.024201 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:51.038462 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 581/1175] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:26:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:26:51.062861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced278a-abb4-4908-9ae5-58789093ec29 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:51.162208 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced278a-abb4-4908-9ae5-58789093ec29 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:51.173023 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:51.196395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a3faa3-75f6-43ab-8f39-e98725b4f18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:51.197291 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3faa3-75f6-43ab-8f39-e98725b4f18c None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:51.197530 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3faa3-75f6-43ab-8f39-e98725b4f18c None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:51.197808 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3faa3-75f6-43ab-8f39-e98725b4f18c None admin] PATH_INFO: `/v3/domains/b5ab6e94c4a34e8f8ca7f015b4d4a23c` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:51.199004 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a3faa3-75f6-43ab-8f39-e98725b4f18c None admin] RBAC: Authorizing `identity:get_domain(domain_id=b5ab6e94c4a34e8f8ca7f015b4d4a23c)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:51.200425 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a3faa3-75f6-43ab-8f39-e98725b4f18c None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:51.202028 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 595/1176] 10.222.0.15 () {56 vars in 1183 bytes} [Wed Jun 15 19:26:51 2022] GET /identity/v3/domains/b5ab6e94c4a34e8f8ca7f015b4d4a23c => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:51.207615 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2526cd1-77d7-4d01-b399-dc044487526a None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:51.302283 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2526cd1-77d7-4d01-b399-dc044487526a None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:51.335537 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba25f0-f3c0-4c06-b3f6-930bb5daeaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:51.336206 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba25f0-f3c0-4c06-b3f6-930bb5daeaf9 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:51.336490 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba25f0-f3c0-4c06-b3f6-930bb5daeaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:51.336768 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba25f0-f3c0-4c06-b3f6-930bb5daeaf9 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:51.337478 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ba25f0-f3c0-4c06-b3f6-930bb5daeaf9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:51.338936 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ba25f0-f3c0-4c06-b3f6-930bb5daeaf9 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:51.357710 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 582/1177] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Jun 15 19:26:51 2022] POST /identity/v3/users => generated 311 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:26:53.489187 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b622e7-dae6-4388-a24e-4eddb00cbf2a None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:53.489763 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b622e7-dae6-4388-a24e-4eddb00cbf2a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:53.489864 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b622e7-dae6-4388-a24e-4eddb00cbf2a None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:53.491237 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 596/1178] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:26:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:26:53.496248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268d12d-e0cd-481d-8cc6-53f35abc572b None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:53.496623 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268d12d-e0cd-481d-8cc6-53f35abc572b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:53.496807 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268d12d-e0cd-481d-8cc6-53f35abc572b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:53.513820 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-1268d12d-e0cd-481d-8cc6-53f35abc572b None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:26:53.515036 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1268d12d-e0cd-481d-8cc6-53f35abc572b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:26:53.553026 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:53.580094 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 583/1179] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:26:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:26:53.603342 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b622e7-dae6-4388-a24e-4eddb00cbf2a None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:53.701836 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b622e7-dae6-4388-a24e-4eddb00cbf2a None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:53.734330 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a2484-a08b-42ab-ac68-5327a1eaa086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:53.735102 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a2484-a08b-42ab-ac68-5327a1eaa086 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:53.735385 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a2484-a08b-42ab-ac68-5327a1eaa086 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:53.735662 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a2484-a08b-42ab-ac68-5327a1eaa086 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:53.746039 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a2484-a08b-42ab-ac68-5327a1eaa086 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:26:53.753591 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-e32a2484-a08b-42ab-ac68-5327a1eaa086 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:26:53.753591 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:26:53.753591 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:26:53.753591 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:26:53.753591 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:26:53.753591 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:26:53.753591 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:26:53.753591 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:26:53.753591 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:26:53.753591 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:26:53.753591 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:26:53.753591 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:26:53.753591 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:26:53.753591 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:26:53.753591 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:26:53.753591 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:26:53.754774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:26:53.754774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:26:53.754774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:26:53.754774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:26:53.754774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:26:53.754774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:26:53.754774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:26:53.754774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:26:53.754774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:26:53.754774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:26:53.754774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:26:53.754774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:26:53.754774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:26:53.754774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:26:53.754774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:26:53.754774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:26:53.756693 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:26:53.756693 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:26:53.756693 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:26:53.756693 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:26:53.756693 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:26:53.756693 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:26:53.756693 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:26:53.756693 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:26:53.756693 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:26:53.756693 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:26:53.756693 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:26:53.756693 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:26:53.756693 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 597/1180] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jun 15 19:26:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:26:53.761409 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1268d12d-e0cd-481d-8cc6-53f35abc572b None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:53.855546 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1268d12d-e0cd-481d-8cc6-53f35abc572b None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:53.888815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a844e3ab-a7ce-4f0b-9e05-076368926b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:53.889463 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844e3ab-a7ce-4f0b-9e05-076368926b83 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:53.889753 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844e3ab-a7ce-4f0b-9e05-076368926b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:53.890030 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844e3ab-a7ce-4f0b-9e05-076368926b83 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:53.890800 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a844e3ab-a7ce-4f0b-9e05-076368926b83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:53.891104 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a844e3ab-a7ce-4f0b-9e05-076368926b83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:53.892464 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a844e3ab-a7ce-4f0b-9e05-076368926b83 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:53.900256 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 584/1181] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:26:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:53.905802 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a2484-a08b-42ab-ac68-5327a1eaa086 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:53.944767 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:54.001980 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a2484-a08b-42ab-ac68-5327a1eaa086 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:54.034852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f58f3-60a1-4ce0-bdb9-d9021d85c5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:54.035707 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f58f3-60a1-4ce0-bdb9-d9021d85c5e3 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:54.035988 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f58f3-60a1-4ce0-bdb9-d9021d85c5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:54.036263 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f58f3-60a1-4ce0-bdb9-d9021d85c5e3 None admin] PATH_INFO: `/v3/users/08537a9657af4d9daa3d0a020b3e1c86` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:54.048778 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16f58f3-60a1-4ce0-bdb9-d9021d85c5e3 None admin] RBAC: Authorizing `identity:get_user(user_id=08537a9657af4d9daa3d0a020b3e1c86)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:54.050363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16f58f3-60a1-4ce0-bdb9-d9021d85c5e3 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:54.053333 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 598/1182] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:26:53 2022] GET /identity/v3/users/08537a9657af4d9daa3d0a020b3e1c86 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:54.059094 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a844e3ab-a7ce-4f0b-9e05-076368926b83 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:54.100510 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:54.163087 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a844e3ab-a7ce-4f0b-9e05-076368926b83 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:54.195516 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259cfb6c-b577-49a6-a738-7d18aa96b302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:54.196144 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259cfb6c-b577-49a6-a738-7d18aa96b302 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:54.196516 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259cfb6c-b577-49a6-a738-7d18aa96b302 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:54.196791 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259cfb6c-b577-49a6-a738-7d18aa96b302 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:54.262810 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259cfb6c-b577-49a6-a738-7d18aa96b302 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:54.264248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259cfb6c-b577-49a6-a738-7d18aa96b302 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:54.297511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:54.362508 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 585/1183] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:26:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 19:26:54.369037 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f58f3-60a1-4ce0-bdb9-d9021d85c5e3 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:54.466362 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:54.472314 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f58f3-60a1-4ce0-bdb9-d9021d85c5e3 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:54.505311 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0327bdb-080e-4834-bf43-9bdec17eb053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:54.506187 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0327bdb-080e-4834-bf43-9bdec17eb053 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:54.506729 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0327bdb-080e-4834-bf43-9bdec17eb053 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:54.507005 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0327bdb-080e-4834-bf43-9bdec17eb053 None admin] PATH_INFO: `/v3/projects/cd59a12596a341a09b5e8b28f74bea6e` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:54.508176 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0327bdb-080e-4834-bf43-9bdec17eb053 None admin] RBAC: Authorizing `identity:get_project(project_id=cd59a12596a341a09b5e8b28f74bea6e)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:54.509595 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0327bdb-080e-4834-bf43-9bdec17eb053 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:54.511320 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 599/1184] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:26:54 2022] GET /identity/v3/projects/cd59a12596a341a09b5e8b28f74bea6e => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:54.517515 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259cfb6c-b577-49a6-a738-7d18aa96b302 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:54.615118 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259cfb6c-b577-49a6-a738-7d18aa96b302 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:54.648504 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e12a9c-9b1a-414e-8f35-38542841969b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:54.649141 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e12a9c-9b1a-414e-8f35-38542841969b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:54.649420 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e12a9c-9b1a-414e-8f35-38542841969b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:54.649693 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e12a9c-9b1a-414e-8f35-38542841969b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:54.650548 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e12a9c-9b1a-414e-8f35-38542841969b None admin] RBAC: Adding query filter params (role.id=4f736313c26d4f8db48bbb2f346e50e4, scope.project.id=cd59a12596a341a09b5e8b28f74bea6e, user.id=08537a9657af4d9daa3d0a020b3e1c86) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:54.650858 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e12a9c-9b1a-414e-8f35-38542841969b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:54.652280 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e12a9c-9b1a-414e-8f35-38542841969b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:54.655104 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:54.658875 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 586/1185] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:26:54 2022] GET /identity/v3/role_assignments?user.id=08537a9657af4d9daa3d0a020b3e1c86&scope.project.id=cd59a12596a341a09b5e8b28f74bea6e&role.id=4f736313c26d4f8db48bbb2f346e50e4 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:56.736567 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba2f27-c711-47ac-8848-d574ee011033 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:56.737079 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba2f27-c711-47ac-8848-d574ee011033 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:56.737079 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba2f27-c711-47ac-8848-d574ee011033 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:56.738461 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 600/1186] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:26:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:26:56.744210 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e4269-bbea-45ed-a478-685b274edbb9 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:56.744636 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e4269-bbea-45ed-a478-685b274edbb9 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:56.744765 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e4269-bbea-45ed-a478-685b274edbb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:56.761905 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-091e4269-bbea-45ed-a478-685b274edbb9 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:26:56.763148 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-091e4269-bbea-45ed-a478-685b274edbb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:26:56.826246 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 587/1187] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:26:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:26:56.848030 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba2f27-c711-47ac-8848-d574ee011033 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:56.950427 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba2f27-c711-47ac-8848-d574ee011033 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:56.981886 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f41c0-5aeb-41e8-b046-44a6ad2a86ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:56.982686 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f41c0-5aeb-41e8-b046-44a6ad2a86ad None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:56.982968 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f41c0-5aeb-41e8-b046-44a6ad2a86ad None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:56.983240 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f41c0-5aeb-41e8-b046-44a6ad2a86ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:56.993420 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f41c0-5aeb-41e8-b046-44a6ad2a86ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:26:57.000928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-927f41c0-5aeb-41e8-b046-44a6ad2a86ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:26:57.000928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:26:57.000928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:26:57.000928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:26:57.000928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:26:57.000928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:26:57.000928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:26:57.000928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:26:57.000928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:26:57.000928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:26:57.000928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:26:57.000928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:26:57.000928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:26:57.000928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:26:57.000928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:26:57.000928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:26:57.002036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:26:57.002036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:26:57.002036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:26:57.002036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:26:57.002036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:26:57.002036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:26:57.002036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:26:57.002036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:26:57.002036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:26:57.002036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:26:57.002036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:26:57.002036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:26:57.002036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:26:57.002036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:26:57.002036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:26:57.002036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:26:57.003185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:26:57.003185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:26:57.003185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:26:57.003185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:26:57.003185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:26:57.003185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:26:57.003185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:26:57.003185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:26:57.003185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:26:57.003185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:26:57.003185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:26:57.003185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:26:57.003185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 601/1188] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jun 15 19:26:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:26:57.009154 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e4269-bbea-45ed-a478-685b274edbb9 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:57.104929 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e4269-bbea-45ed-a478-685b274edbb9 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:57.132778 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:57.138769 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa0a59d-5836-4a12-aa73-82cacffb3b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:57.139424 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0a59d-5836-4a12-aa73-82cacffb3b6d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:57.139705 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0a59d-5836-4a12-aa73-82cacffb3b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:57.139980 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0a59d-5836-4a12-aa73-82cacffb3b6d None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:57.140724 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa0a59d-5836-4a12-aa73-82cacffb3b6d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:26:57.141029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa0a59d-5836-4a12-aa73-82cacffb3b6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:57.142419 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa0a59d-5836-4a12-aa73-82cacffb3b6d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:57.150359 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 588/1189] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:26:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:57.156042 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f41c0-5aeb-41e8-b046-44a6ad2a86ad None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:57.181707 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:57.260358 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f41c0-5aeb-41e8-b046-44a6ad2a86ad None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:57.292133 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c463bd-e5f4-453a-bd7b-b9f33d5baebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:57.292993 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c463bd-e5f4-453a-bd7b-b9f33d5baebd None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:57.293271 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c463bd-e5f4-453a-bd7b-b9f33d5baebd None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:57.293540 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c463bd-e5f4-453a-bd7b-b9f33d5baebd None admin] PATH_INFO: `/v3/users/08537a9657af4d9daa3d0a020b3e1c86` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:57.294720 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c463bd-e5f4-453a-bd7b-b9f33d5baebd None admin] RBAC: Authorizing `identity:get_user(user_id=08537a9657af4d9daa3d0a020b3e1c86)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:57.296134 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c463bd-e5f4-453a-bd7b-b9f33d5baebd None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:57.297741 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 602/1190] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:26:57 2022] GET /identity/v3/users/08537a9657af4d9daa3d0a020b3e1c86 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:57.304215 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa0a59d-5836-4a12-aa73-82cacffb3b6d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:57.400680 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa0a59d-5836-4a12-aa73-82cacffb3b6d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:57.433289 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e6fb23-c963-49b9-a315-02578a5b6dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:57.434167 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e6fb23-c963-49b9-a315-02578a5b6dde None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:57.434455 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e6fb23-c963-49b9-a315-02578a5b6dde None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:57.434733 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e6fb23-c963-49b9-a315-02578a5b6dde None admin] PATH_INFO: `/v3/projects/cd59a12596a341a09b5e8b28f74bea6e` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:57.435914 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e6fb23-c963-49b9-a315-02578a5b6dde None admin] RBAC: Authorizing `identity:get_project(project_id=cd59a12596a341a09b5e8b28f74bea6e)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:57.437343 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e6fb23-c963-49b9-a315-02578a5b6dde None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:57.439066 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 589/1191] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:26:57 2022] GET /identity/v3/projects/cd59a12596a341a09b5e8b28f74bea6e => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:26:57.444981 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c463bd-e5f4-453a-bd7b-b9f33d5baebd None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:57.533027 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:57.541481 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c463bd-e5f4-453a-bd7b-b9f33d5baebd None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:57.574101 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25283606-b6fa-4f28-93f7-f1e8de4dfc86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:57.574990 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25283606-b6fa-4f28-93f7-f1e8de4dfc86 None admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:57.575272 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25283606-b6fa-4f28-93f7-f1e8de4dfc86 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:57.575546 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25283606-b6fa-4f28-93f7-f1e8de4dfc86 None admin] PATH_INFO: `/v3/projects/cd59a12596a341a09b5e8b28f74bea6e/users/08537a9657af4d9daa3d0a020b3e1c86/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:57.577474 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25283606-b6fa-4f28-93f7-f1e8de4dfc86 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd59a12596a341a09b5e8b28f74bea6e, user_id=08537a9657af4d9daa3d0a020b3e1c86, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:26:57.578929 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25283606-b6fa-4f28-93f7-f1e8de4dfc86 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:26:57.590798 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 603/1192] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jun 15 19:26:57 2022] PUT /identity/v3/projects/cd59a12596a341a09b5e8b28f74bea6e/users/08537a9657af4d9daa3d0a020b3e1c86/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:26:59.671788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b57ab2-d688-423f-897a-1d41c6c4166a None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:59.672291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b57ab2-d688-423f-897a-1d41c6c4166a None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:59.672291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b57ab2-d688-423f-897a-1d41c6c4166a None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:59.673632 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 590/1193] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:26:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:26:59.678810 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc499aa-57b6-4c89-9dae-b2deaf9877c1 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:59.679227 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc499aa-57b6-4c89-9dae-b2deaf9877c1 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:59.679340 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc499aa-57b6-4c89-9dae-b2deaf9877c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:59.695992 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-3fc499aa-57b6-4c89-9dae-b2deaf9877c1 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:26:59.697167 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-3fc499aa-57b6-4c89-9dae-b2deaf9877c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:26:59.760589 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 604/1194] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:26:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:26:59.783518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b57ab2-d688-423f-897a-1d41c6c4166a None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:26:59.842604 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:26:59.883689 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b57ab2-d688-423f-897a-1d41c6c4166a None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:26:59.916383 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc129e4-17da-489f-bee1-08ed211872d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:26:59.917160 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc129e4-17da-489f-bee1-08ed211872d4 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:26:59.917444 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc129e4-17da-489f-bee1-08ed211872d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:26:59.917721 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc129e4-17da-489f-bee1-08ed211872d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:26:59.928288 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc129e4-17da-489f-bee1-08ed211872d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:26:59.935817 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-7cc129e4-17da-489f-bee1-08ed211872d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:26:59.935817 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:26:59.935817 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:26:59.935817 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:26:59.935817 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:26:59.935817 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:26:59.935817 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:26:59.935817 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:26:59.935817 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:26:59.935817 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:26:59.935817 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:26:59.935817 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:26:59.935817 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:26:59.935817 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:26:59.935817 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:26:59.935817 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:26:59.936912 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:26:59.936912 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:26:59.936912 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:26:59.936912 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:26:59.936912 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:26:59.936912 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:26:59.936912 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:26:59.936912 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:26:59.936912 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:26:59.936912 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:26:59.936912 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:26:59.936912 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:26:59.936912 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:26:59.936912 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:26:59.936912 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:26:59.936912 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:26:59.938140 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:26:59.938140 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:26:59.938140 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:26:59.938140 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:26:59.938140 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:26:59.938140 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:26:59.938140 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:26:59.938140 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:26:59.938140 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:26:59.938140 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:26:59.938140 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:26:59.938140 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:26:59.938140 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 591/1195] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jun 15 19:26:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:26:59.942979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc499aa-57b6-4c89-9dae-b2deaf9877c1 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:27:00.020115 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:27:00.039481 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc499aa-57b6-4c89-9dae-b2deaf9877c1 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:27:00.071733 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ce550-76f1-4b98-ac7d-ce8fcb13cb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:27:00.072398 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ce550-76f1-4b98-ac7d-ce8fcb13cb2c None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:27:00.072685 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ce550-76f1-4b98-ac7d-ce8fcb13cb2c None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:27:00.072965 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ce550-76f1-4b98-ac7d-ce8fcb13cb2c None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:27:00.073711 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ce550-76f1-4b98-ac7d-ce8fcb13cb2c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:27:00.074019 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ce550-76f1-4b98-ac7d-ce8fcb13cb2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:27:00.075414 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ce550-76f1-4b98-ac7d-ce8fcb13cb2c None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:27:00.084392 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 605/1196] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:26:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:27:00.090389 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc129e4-17da-489f-bee1-08ed211872d4 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:27:00.190860 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc129e4-17da-489f-bee1-08ed211872d4 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:27:00.212086 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:27:00.225643 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa1fe8-36ba-4843-9c6b-7cb4932605c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:27:00.226521 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa1fe8-36ba-4843-9c6b-7cb4932605c5 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:27:00.226807 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa1fe8-36ba-4843-9c6b-7cb4932605c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:27:00.227083 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa1fe8-36ba-4843-9c6b-7cb4932605c5 None admin] PATH_INFO: `/v3/users/08537a9657af4d9daa3d0a020b3e1c86` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:27:00.228268 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa1fe8-36ba-4843-9c6b-7cb4932605c5 None admin] RBAC: Authorizing `identity:get_user(user_id=08537a9657af4d9daa3d0a020b3e1c86)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:27:00.229695 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa1fe8-36ba-4843-9c6b-7cb4932605c5 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:27:00.231374 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 592/1197] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Jun 15 19:27:00 2022] GET /identity/v3/users/08537a9657af4d9daa3d0a020b3e1c86 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:27:00.237421 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ce550-76f1-4b98-ac7d-ce8fcb13cb2c None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:27:00.335097 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ce550-76f1-4b98-ac7d-ce8fcb13cb2c None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:27:00.367432 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61103d-09a8-4519-b5e5-16d6fe6ee698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:27:00.368073 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61103d-09a8-4519-b5e5-16d6fe6ee698 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:27:00.368360 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61103d-09a8-4519-b5e5-16d6fe6ee698 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:27:00.368639 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61103d-09a8-4519-b5e5-16d6fe6ee698 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:27:00.383122 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:27:00.435347 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c61103d-09a8-4519-b5e5-16d6fe6ee698 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:27:00.436790 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c61103d-09a8-4519-b5e5-16d6fe6ee698 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:27:00.530954 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 606/1198] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:27:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 19:27:00.537405 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa1fe8-36ba-4843-9c6b-7cb4932605c5 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:27:00.633244 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa1fe8-36ba-4843-9c6b-7cb4932605c5 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:27:00.666460 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64db5333-a591-43e3-8f87-6b4d1bd701cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:27:00.667304 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db5333-a591-43e3-8f87-6b4d1bd701cd None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:27:00.667584 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db5333-a591-43e3-8f87-6b4d1bd701cd None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:27:00.667856 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db5333-a591-43e3-8f87-6b4d1bd701cd None admin] PATH_INFO: `/v3/projects/cd59a12596a341a09b5e8b28f74bea6e` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:27:00.669031 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64db5333-a591-43e3-8f87-6b4d1bd701cd None admin] RBAC: Authorizing `identity:get_project(project_id=cd59a12596a341a09b5e8b28f74bea6e)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:27:00.670480 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64db5333-a591-43e3-8f87-6b4d1bd701cd None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:27:00.672131 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 593/1199] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Jun 15 19:27:00 2022] GET /identity/v3/projects/cd59a12596a341a09b5e8b28f74bea6e => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:27:00.678018 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61103d-09a8-4519-b5e5-16d6fe6ee698 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:27:00.722278 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:27:00.774747 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61103d-09a8-4519-b5e5-16d6fe6ee698 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:27:00.807893 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297c2c48-6b4d-43d8-914e-c3a9f72ec944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:27:00.808480 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c2c48-6b4d-43d8-914e-c3a9f72ec944 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:27:00.808767 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c2c48-6b4d-43d8-914e-c3a9f72ec944 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:27:00.809046 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c2c48-6b4d-43d8-914e-c3a9f72ec944 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:27:00.809870 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297c2c48-6b4d-43d8-914e-c3a9f72ec944 None admin] RBAC: Adding query filter params (role.id=4f736313c26d4f8db48bbb2f346e50e4, scope.project.id=cd59a12596a341a09b5e8b28f74bea6e, user.id=08537a9657af4d9daa3d0a020b3e1c86) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:27:00.810214 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297c2c48-6b4d-43d8-914e-c3a9f72ec944 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:27:00.811632 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297c2c48-6b4d-43d8-914e-c3a9f72ec944 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:27:00.817494 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 607/1200] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:27:00 2022] GET /identity/v3/role_assignments?user.id=08537a9657af4d9daa3d0a020b3e1c86&scope.project.id=cd59a12596a341a09b5e8b28f74bea6e&role.id=4f736313c26d4f8db48bbb2f346e50e4 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:27:10.662534 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=72 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:27:10.814735 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=46 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:27:56.241069 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab9337-705a-47db-8c11-03b505a7a3f2 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:27:56.241069 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab9337-705a-47db-8c11-03b505a7a3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:27:56.241069 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab9337-705a-47db-8c11-03b505a7a3f2 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:27:56.242392 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 594/1201] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:27:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:27:56.246994 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f23fe-5c34-4b76-8875-a82410aa3aad None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:27:56.247472 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f23fe-5c34-4b76-8875-a82410aa3aad None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:27:56.247559 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f23fe-5c34-4b76-8875-a82410aa3aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:27:56.267847 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-0c2f23fe-5c34-4b76-8875-a82410aa3aad None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:27:56.269577 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-0c2f23fe-5c34-4b76-8875-a82410aa3aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:27:56.367100 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 608/1202] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:27:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 19:27:56.395790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab9337-705a-47db-8c11-03b505a7a3f2 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:27:56.551458 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab9337-705a-47db-8c11-03b505a7a3f2 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:27:56.599917 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90b8d7-4765-40f5-865d-35315010ac47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:27:56.600823 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90b8d7-4765-40f5-865d-35315010ac47 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:27:56.601089 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90b8d7-4765-40f5-865d-35315010ac47 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:27:56.601345 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90b8d7-4765-40f5-865d-35315010ac47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:27:56.619733 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f90b8d7-4765-40f5-865d-35315010ac47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:27:56.621211 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f90b8d7-4765-40f5-865d-35315010ac47 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:27:56.648739 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-0f90b8d7-4765-40f5-865d-35315010ac47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:27:56.648739 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:27:56.648739 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:27:56.648739 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:27:56.648739 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:27:56.648739 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:27:56.648739 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:27:56.648739 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:27:56.648739 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:27:56.648739 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:27:56.648739 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:27:56.648739 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:27:56.648739 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:27:56.648739 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:27:56.648739 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:27:56.650023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:27:56.650023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:27:56.650023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:27:56.650023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:27:56.650023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:27:56.650023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:27:56.650023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:27:56.650023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:27:56.650023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:27:56.650023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:27:56.650023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:27:56.650023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:27:56.650023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:27:56.650023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:27:56.650023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:27:56.650023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:27:56.650023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:27:56.651528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:27:56.651528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:27:56.651528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:27:56.651528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:27:56.651528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 595/1203] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:27:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:27:56.656856 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f23fe-5c34-4b76-8875-a82410aa3aad None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:27:56.805507 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f23fe-5c34-4b76-8875-a82410aa3aad None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:27:56.844661 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:27:56.852272 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399ac558-be6a-4239-a117-256b8c25fc86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:27:56.853230 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ac558-be6a-4239-a117-256b8c25fc86 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:27:56.853719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ac558-be6a-4239-a117-256b8c25fc86 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:27:56.854271 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ac558-be6a-4239-a117-256b8c25fc86 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:27:56.855211 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399ac558-be6a-4239-a117-256b8c25fc86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:27:56.855527 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399ac558-be6a-4239-a117-256b8c25fc86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:27:56.856983 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399ac558-be6a-4239-a117-256b8c25fc86 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:27:56.870754 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 609/1204] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:27:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:27:56.876416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90b8d7-4765-40f5-865d-35315010ac47 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:27:56.949865 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:27:57.016420 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90b8d7-4765-40f5-865d-35315010ac47 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:27:57.074661 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6deb6aa-a6ea-4bb5-9f07-232b43e8c25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:27:57.075330 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6deb6aa-a6ea-4bb5-9f07-232b43e8c25e None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:27:57.075678 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6deb6aa-a6ea-4bb5-9f07-232b43e8c25e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:27:57.075987 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6deb6aa-a6ea-4bb5-9f07-232b43e8c25e None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:27:57.076819 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6deb6aa-a6ea-4bb5-9f07-232b43e8c25e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:27:57.078581 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6deb6aa-a6ea-4bb5-9f07-232b43e8c25e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:27:57.112493 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 596/1205] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Jun 15 19:27:56 2022] POST /identity/v3/users => generated 251 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:27:59.420763 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ae0aa-5cdf-43b8-9e3f-c94384568db9 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:27:59.421368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ae0aa-5cdf-43b8-9e3f-c94384568db9 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:27:59.421368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ae0aa-5cdf-43b8-9e3f-c94384568db9 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:27:59.423389 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 610/1206] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:27:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:27:59.430028 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d19ce-d94e-47f6-990e-163d5e73af9f None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:27:59.430585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d19ce-d94e-47f6-990e-163d5e73af9f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:27:59.431023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d19ce-d94e-47f6-990e-163d5e73af9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:27:59.449744 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c68d19ce-d94e-47f6-990e-163d5e73af9f None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:27:59.451119 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c68d19ce-d94e-47f6-990e-163d5e73af9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:27:59.534511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 597/1207] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:27:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:27:59.555549 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ae0aa-5cdf-43b8-9e3f-c94384568db9 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:27:59.675403 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ae0aa-5cdf-43b8-9e3f-c94384568db9 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:27:59.716788 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb3f02-33a9-4657-9cb1-343bdce78903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:27:59.717593 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb3f02-33a9-4657-9cb1-343bdce78903 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:27:59.717878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb3f02-33a9-4657-9cb1-343bdce78903 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:27:59.718196 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb3f02-33a9-4657-9cb1-343bdce78903 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:27:59.730017 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb3f02-33a9-4657-9cb1-343bdce78903 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:27:59.738480 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-cadb3f02-33a9-4657-9cb1-343bdce78903 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:27:59.738480 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:27:59.738480 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:27:59.738480 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:27:59.738480 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:27:59.738480 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:27:59.738480 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:27:59.738480 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:27:59.738480 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:27:59.738480 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:27:59.738480 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:27:59.738480 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:27:59.738480 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:27:59.738480 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:27:59.738480 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:27:59.738480 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:27:59.739563 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:27:59.739563 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:27:59.739563 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:27:59.739563 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:27:59.739563 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:27:59.739563 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:27:59.739563 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:27:59.739563 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:27:59.739563 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:27:59.739563 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:27:59.739563 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:27:59.739563 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:27:59.739563 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:27:59.739563 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:27:59.739563 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:27:59.739563 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:27:59.743253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:27:59.743253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:27:59.743253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:27:59.743253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:27:59.743253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:27:59.743253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:27:59.743253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:27:59.743253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:27:59.743253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:27:59.743253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:27:59.743253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:27:59.743253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:27:59.743253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 611/1208] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jun 15 19:27:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:27:59.745768 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d19ce-d94e-47f6-990e-163d5e73af9f None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:27:59.791422 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:27:59.855886 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d19ce-d94e-47f6-990e-163d5e73af9f None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:27:59.894031 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4fab6-0ffb-460c-a199-003351a51de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:27:59.894772 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4fab6-0ffb-460c-a199-003351a51de9 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:27:59.895166 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4fab6-0ffb-460c-a199-003351a51de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:27:59.895420 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4fab6-0ffb-460c-a199-003351a51de9 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:27:59.896289 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae4fab6-0ffb-460c-a199-003351a51de9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:27:59.896603 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae4fab6-0ffb-460c-a199-003351a51de9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:27:59.897974 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae4fab6-0ffb-460c-a199-003351a51de9 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:27:59.906618 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 598/1209] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jun 15 19:27:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:27:59.914174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb3f02-33a9-4657-9cb1-343bdce78903 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:27:59.973039 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:00.030354 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb3f02-33a9-4657-9cb1-343bdce78903 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:00.069673 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa802e4-0bde-4b7d-ab1f-6bb4b0b08e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:00.070517 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa802e4-0bde-4b7d-ab1f-6bb4b0b08e2f None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:00.070798 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa802e4-0bde-4b7d-ab1f-6bb4b0b08e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:00.071079 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa802e4-0bde-4b7d-ab1f-6bb4b0b08e2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:00.082889 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa802e4-0bde-4b7d-ab1f-6bb4b0b08e2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:00.084490 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa802e4-0bde-4b7d-ab1f-6bb4b0b08e2f None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:00.104352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-9aa802e4-0bde-4b7d-ab1f-6bb4b0b08e2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:28:00.104352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:28:00.104352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:28:00.104352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:28:00.104352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:28:00.104352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:28:00.104352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:28:00.104352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:28:00.104352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:28:00.104352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:28:00.104352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:28:00.104352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:28:00.104352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:28:00.104352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:28:00.104352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:28:00.105719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:28:00.105719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:28:00.105719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:28:00.105719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:28:00.105719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:28:00.105719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:28:00.105719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:28:00.105719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:28:00.105719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:28:00.105719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:28:00.105719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:28:00.105719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:28:00.105719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:28:00.105719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:28:00.105719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:28:00.105719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:28:00.105719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:28:00.113117 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:28:00.113117 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:28:00.113117 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:28:00.113117 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:28:00.113117 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 612/1210] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:27:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:28:00.113953 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4fab6-0ffb-460c-a199-003351a51de9 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:00.228026 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:00.231676 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4fab6-0ffb-460c-a199-003351a51de9 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:00.270922 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29714a31-585b-4396-8f8b-8ae8e108a819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:00.271586 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29714a31-585b-4396-8f8b-8ae8e108a819 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:00.271866 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29714a31-585b-4396-8f8b-8ae8e108a819 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:00.272141 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29714a31-585b-4396-8f8b-8ae8e108a819 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:00.272865 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29714a31-585b-4396-8f8b-8ae8e108a819 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:00.273169 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29714a31-585b-4396-8f8b-8ae8e108a819 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:00.274584 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29714a31-585b-4396-8f8b-8ae8e108a819 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:00.287553 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 599/1211] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:28:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:28:00.293238 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa802e4-0bde-4b7d-ab1f-6bb4b0b08e2f None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:00.429624 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa802e4-0bde-4b7d-ab1f-6bb4b0b08e2f None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:00.431490 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:00.472134 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2611c3b8-23b9-4032-9817-650855c8d8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:00.473130 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611c3b8-23b9-4032-9817-650855c8d8cb None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:00.473415 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611c3b8-23b9-4032-9817-650855c8d8cb None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:00.473765 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611c3b8-23b9-4032-9817-650855c8d8cb None admin] PATH_INFO: `/v3/users/placement` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:00.484443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2611c3b8-23b9-4032-9817-650855c8d8cb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:00.484443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2611c3b8-23b9-4032-9817-650855c8d8cb None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:00.501613 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-2611c3b8-23b9-4032-9817-650855c8d8cb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 19:28:00.501613 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:28:00.501613 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:28:00.501613 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:28:00.501613 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:28:00.501613 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:28:00.501613 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:28:00.501613 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:28:00.501613 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:28:00.501613 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:28:00.501613 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:28:00.501613 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:28:00.501613 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 19:28:00.501613 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 19:28:00.501613 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 19:28:00.501613 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 19:28:00.503710 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:28:00.503710 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:28:00.503710 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:28:00.503710 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:28:00.503710 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:28:00.503710 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:28:00.503710 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:28:00.503710 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:28:00.503710 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:28:00.503710 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:28:00.503710 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:28:00.503710 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:28:00.503710 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:28:00.503710 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:28:00.503710 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:28:00.503710 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:28:00.503710 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:28:00.503710 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:28:00.505107 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:28:00.505107 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:28:00.505107 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 19:28:00.505107 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 19:28:00.505107 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 19:28:00.505107 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 19:28:00.505107 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 19:28:00.505107 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 19:28:00.505107 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 19:28:00.505107 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:28:00.505107 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 613/1212] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:28:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:28:00.510036 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29714a31-585b-4396-8f8b-8ae8e108a819 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:00.621004 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29714a31-585b-4396-8f8b-8ae8e108a819 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:00.659910 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f9f679-befd-4516-a97f-3256fc90d615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:00.661014 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f9f679-befd-4516-a97f-3256fc90d615 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:00.661507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f9f679-befd-4516-a97f-3256fc90d615 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:00.662003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f9f679-befd-4516-a97f-3256fc90d615 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:00.663092 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f9f679-befd-4516-a97f-3256fc90d615 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:00.664879 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f9f679-befd-4516-a97f-3256fc90d615 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:00.665221 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f9f679-befd-4516-a97f-3256fc90d615 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:00.686369 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 600/1213] 10.222.0.15 () {56 vars in 1143 bytes} [Wed Jun 15 19:28:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:28:00.692167 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2611c3b8-23b9-4032-9817-650855c8d8cb None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:00.813435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2611c3b8-23b9-4032-9817-650855c8d8cb None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:00.854603 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a058f-19ae-4d46-86b7-484e4f8c6b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:00.855674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a058f-19ae-4d46-86b7-484e4f8c6b41 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:00.855674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a058f-19ae-4d46-86b7-484e4f8c6b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:00.855978 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a058f-19ae-4d46-86b7-484e4f8c6b41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:00.888733 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:00.930476 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a058f-19ae-4d46-86b7-484e4f8c6b41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:00.932040 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a058f-19ae-4d46-86b7-484e4f8c6b41 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:01.046131 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 614/1214] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:28:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 19:28:01.047553 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f9f679-befd-4516-a97f-3256fc90d615 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:01.076620 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:01.164440 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f9f679-befd-4516-a97f-3256fc90d615 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:01.205164 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0971ea42-44ca-4b9b-8446-a766be88bd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:01.206690 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971ea42-44ca-4b9b-8446-a766be88bd32 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:01.206977 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971ea42-44ca-4b9b-8446-a766be88bd32 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:01.207359 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971ea42-44ca-4b9b-8446-a766be88bd32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:01.218595 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0971ea42-44ca-4b9b-8446-a766be88bd32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:01.220177 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0971ea42-44ca-4b9b-8446-a766be88bd32 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:01.241674 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-0971ea42-44ca-4b9b-8446-a766be88bd32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:28:01.241674 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:28:01.241674 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:28:01.241674 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:28:01.241674 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:28:01.241674 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:28:01.241674 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:28:01.241674 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:28:01.241674 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:28:01.241674 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:28:01.241674 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:28:01.241674 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:28:01.241674 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:28:01.241674 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:28:01.241674 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:28:01.242729 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:28:01.242729 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:28:01.242729 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:28:01.242729 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:28:01.242729 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:28:01.242729 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:28:01.242729 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:28:01.242729 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:28:01.242729 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:28:01.242729 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:28:01.242729 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:28:01.242729 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:28:01.242729 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:28:01.242729 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:28:01.242729 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:28:01.242729 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:28:01.242729 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:28:01.243639 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:28:01.243639 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:28:01.243639 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:28:01.243639 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:28:01.243639 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 601/1215] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:28:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:28:01.248728 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a058f-19ae-4d46-86b7-484e4f8c6b41 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:01.316402 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:01.360468 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a058f-19ae-4d46-86b7-484e4f8c6b41 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:01.401248 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221baaea-b60e-4a6f-adf6-e9495a01e533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:01.401936 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221baaea-b60e-4a6f-adf6-e9495a01e533 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:01.402258 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221baaea-b60e-4a6f-adf6-e9495a01e533 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:01.402537 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221baaea-b60e-4a6f-adf6-e9495a01e533 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:01.403281 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221baaea-b60e-4a6f-adf6-e9495a01e533 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:01.403594 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221baaea-b60e-4a6f-adf6-e9495a01e533 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:01.405015 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221baaea-b60e-4a6f-adf6-e9495a01e533 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:01.418420 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 615/1216] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:28:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:28:01.424602 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0971ea42-44ca-4b9b-8446-a766be88bd32 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:01.505487 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:01.540439 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0971ea42-44ca-4b9b-8446-a766be88bd32 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:01.590988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fd16f6-ac58-4eea-be66-9ecf93e8bcd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:01.591886 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd16f6-ac58-4eea-be66-9ecf93e8bcd2 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:01.592173 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd16f6-ac58-4eea-be66-9ecf93e8bcd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:01.592462 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd16f6-ac58-4eea-be66-9ecf93e8bcd2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:01.601430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fd16f6-ac58-4eea-be66-9ecf93e8bcd2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:01.602971 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fd16f6-ac58-4eea-be66-9ecf93e8bcd2 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:01.618753 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-88fd16f6-ac58-4eea-be66-9ecf93e8bcd2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:28:01.618753 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:28:01.618753 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:28:01.618753 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:28:01.618753 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:28:01.618753 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:28:01.618753 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:28:01.618753 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:28:01.618753 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:28:01.618753 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:28:01.618753 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:28:01.618753 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:28:01.618753 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 19:28:01.618753 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 19:28:01.618753 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 19:28:01.619854 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 19:28:01.619854 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:28:01.619854 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:28:01.619854 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:28:01.619854 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:28:01.619854 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:28:01.619854 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:28:01.619854 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:28:01.619854 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:28:01.619854 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:28:01.619854 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:28:01.619854 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:28:01.619854 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:28:01.619854 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:28:01.619854 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:28:01.619854 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:28:01.619854 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:28:01.620791 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 19:28:01.620791 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 19:28:01.620791 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:28:01.620791 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:28:01.620791 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:28:01.620791 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:28:01.620791 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:28:01.620791 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:28:01.620791 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 602/1217] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:28:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:28:01.626236 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221baaea-b60e-4a6f-adf6-e9495a01e533 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:01.736711 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221baaea-b60e-4a6f-adf6-e9495a01e533 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:01.764430 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:01.778124 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4feb23b-dad3-4495-973e-98a869622907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:01.778852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4feb23b-dad3-4495-973e-98a869622907 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:01.779135 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4feb23b-dad3-4495-973e-98a869622907 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:01.779408 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4feb23b-dad3-4495-973e-98a869622907 None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:01.780193 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4feb23b-dad3-4495-973e-98a869622907 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:01.780504 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4feb23b-dad3-4495-973e-98a869622907 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:01.781914 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4feb23b-dad3-4495-973e-98a869622907 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:01.795119 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 616/1218] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jun 15 19:28:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:28:01.801669 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fd16f6-ac58-4eea-be66-9ecf93e8bcd2 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:01.920817 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fd16f6-ac58-4eea-be66-9ecf93e8bcd2 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:01.960247 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:01.968780 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3dd412-560e-45fe-8912-9d59d5e875ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:01.969149 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3dd412-560e-45fe-8912-9d59d5e875ea None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:01.969437 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3dd412-560e-45fe-8912-9d59d5e875ea None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:01.969711 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3dd412-560e-45fe-8912-9d59d5e875ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:01.970614 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3dd412-560e-45fe-8912-9d59d5e875ea None admin] RBAC: Adding query filter params (role.id=a8df9714563b435e9fce55a04e74ae5e, scope.project.id=7fb8cc0eb560465e972ffd45867ee73b, user.id=cb0ec2ef6c4748e181f1963e7dfe30b2) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:01.970924 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3dd412-560e-45fe-8912-9d59d5e875ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:01.972475 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3dd412-560e-45fe-8912-9d59d5e875ea None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:01.978834 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 603/1219] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:28:01 2022] GET /identity/v3/role_assignments?user.id=cb0ec2ef6c4748e181f1963e7dfe30b2&scope.project.id=7fb8cc0eb560465e972ffd45867ee73b&role.id=a8df9714563b435e9fce55a04e74ae5e => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:28:04.165539 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce828a-10d6-42a6-9c46-99a3ea2f09ff None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:04.166107 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce828a-10d6-42a6-9c46-99a3ea2f09ff None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:04.166107 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce828a-10d6-42a6-9c46-99a3ea2f09ff None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:04.167531 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 617/1220] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:28:04.173449 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef137d8-ec49-4326-8c7e-12b337983d7f None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:04.173931 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef137d8-ec49-4326-8c7e-12b337983d7f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:04.174033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef137d8-ec49-4326-8c7e-12b337983d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:04.192068 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-2ef137d8-ec49-4326-8c7e-12b337983d7f None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:28:04.193489 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-2ef137d8-ec49-4326-8c7e-12b337983d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:28:04.268667 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 604/1221] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:28:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:28:04.291917 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ce828a-10d6-42a6-9c46-99a3ea2f09ff None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:04.403508 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ce828a-10d6-42a6-9c46-99a3ea2f09ff None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:04.438908 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa87a38-f881-49f4-ba9c-ca5aa3040375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:04.439627 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa87a38-f881-49f4-ba9c-ca5aa3040375 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:04.439989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa87a38-f881-49f4-ba9c-ca5aa3040375 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:04.440773 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa87a38-f881-49f4-ba9c-ca5aa3040375 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:04.451479 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa87a38-f881-49f4-ba9c-ca5aa3040375 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:28:04.459971 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-3fa87a38-f881-49f4-ba9c-ca5aa3040375 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:28:04.459971 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:28:04.459971 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:28:04.459971 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:28:04.459971 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:28:04.459971 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:28:04.459971 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:28:04.459971 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:28:04.459971 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:28:04.459971 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:28:04.459971 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:28:04.459971 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:28:04.459971 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:28:04.459971 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:28:04.459971 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:28:04.459971 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:28:04.461174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:28:04.461174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:28:04.461174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:28:04.461174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:28:04.461174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:28:04.461174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:28:04.461174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:28:04.461174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:28:04.461174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:28:04.461174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:28:04.461174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:28:04.461174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:28:04.461174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:28:04.461174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:28:04.461174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:28:04.461174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:28:04.462376 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:28:04.462376 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:28:04.462376 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:28:04.462376 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:28:04.462376 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:28:04.462376 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:28:04.462376 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:28:04.462376 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:28:04.462376 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:28:04.462376 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:28:04.462376 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:28:04.462376 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:28:04.462376 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 618/1222] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jun 15 19:28:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:28:04.467757 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef137d8-ec49-4326-8c7e-12b337983d7f None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:04.575223 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef137d8-ec49-4326-8c7e-12b337983d7f None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:04.599576 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:04.611771 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67c534-e8db-4cdd-97b6-72e717cb9f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:04.612532 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67c534-e8db-4cdd-97b6-72e717cb9f1d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:04.612854 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67c534-e8db-4cdd-97b6-72e717cb9f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:04.613155 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67c534-e8db-4cdd-97b6-72e717cb9f1d None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:04.613911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67c534-e8db-4cdd-97b6-72e717cb9f1d None admin] RBAC: Adding query filter params (name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:04.614263 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67c534-e8db-4cdd-97b6-72e717cb9f1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:04.615643 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67c534-e8db-4cdd-97b6-72e717cb9f1d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:04.624006 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 605/1223] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jun 15 19:28:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:28:04.630519 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa87a38-f881-49f4-ba9c-ca5aa3040375 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:04.678639 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:04.744024 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa87a38-f881-49f4-ba9c-ca5aa3040375 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:04.781848 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35832c1-d1ac-45f5-986b-c5495c83c56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:04.782672 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35832c1-d1ac-45f5-986b-c5495c83c56f None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:04.782976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35832c1-d1ac-45f5-986b-c5495c83c56f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:04.783257 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35832c1-d1ac-45f5-986b-c5495c83c56f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:04.794654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35832c1-d1ac-45f5-986b-c5495c83c56f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:04.796110 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35832c1-d1ac-45f5-986b-c5495c83c56f None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:04.814315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-c35832c1-d1ac-45f5-986b-c5495c83c56f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:28:04.814315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:28:04.814315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:28:04.814315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:28:04.814315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:28:04.814315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:28:04.814315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:28:04.814315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:28:04.814315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:28:04.814315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:28:04.814315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:28:04.814315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:28:04.814315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:28:04.814315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:28:04.814315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:28:04.815496 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:28:04.815496 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:28:04.815496 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:28:04.815496 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:28:04.815496 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:28:04.815496 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:28:04.815496 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:28:04.815496 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:28:04.815496 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:28:04.815496 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:28:04.815496 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:28:04.815496 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:28:04.815496 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:28:04.815496 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:28:04.815496 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:28:04.815496 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:28:04.815496 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:28:04.816525 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:28:04.816525 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:28:04.816525 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:28:04.816525 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:28:04.816525 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 619/1224] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:28:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:28:04.821964 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67c534-e8db-4cdd-97b6-72e717cb9f1d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:04.936549 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67c534-e8db-4cdd-97b6-72e717cb9f1d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:04.975240 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e701b0f-a269-4b5b-aa6a-0c2ba86b841d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:04.976029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e701b0f-a269-4b5b-aa6a-0c2ba86b841d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:04.976323 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e701b0f-a269-4b5b-aa6a-0c2ba86b841d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:04.976615 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e701b0f-a269-4b5b-aa6a-0c2ba86b841d None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:04.977419 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e701b0f-a269-4b5b-aa6a-0c2ba86b841d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:04.977728 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e701b0f-a269-4b5b-aa6a-0c2ba86b841d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:04.979313 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e701b0f-a269-4b5b-aa6a-0c2ba86b841d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:04.992714 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 606/1225] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:28:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:28:04.998507 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35832c1-d1ac-45f5-986b-c5495c83c56f None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:05.103638 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:05.111388 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35832c1-d1ac-45f5-986b-c5495c83c56f None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:05.148436 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361fdb98-141e-4fea-9280-75f8da76f933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:05.149262 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361fdb98-141e-4fea-9280-75f8da76f933 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:05.149540 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361fdb98-141e-4fea-9280-75f8da76f933 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:05.149811 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361fdb98-141e-4fea-9280-75f8da76f933 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:05.157490 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361fdb98-141e-4fea-9280-75f8da76f933 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:05.158982 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361fdb98-141e-4fea-9280-75f8da76f933 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:05.174036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-361fdb98-141e-4fea-9280-75f8da76f933 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 19:28:05.174036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:28:05.174036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:28:05.174036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:28:05.174036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:28:05.174036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:28:05.174036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:28:05.174036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:28:05.174036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:28:05.174036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:28:05.174036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:28:05.174036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:28:05.174036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 19:28:05.174036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 19:28:05.174036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 19:28:05.174036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 19:28:05.176260 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:28:05.176260 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:28:05.176260 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:28:05.176260 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:28:05.176260 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:28:05.176260 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:28:05.176260 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:28:05.176260 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:28:05.176260 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:28:05.176260 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:28:05.176260 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:28:05.176260 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:28:05.176260 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:28:05.176260 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:28:05.176260 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:28:05.176260 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:28:05.176260 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:28:05.176260 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:28:05.177605 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:28:05.177605 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:28:05.177605 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 19:28:05.177605 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 19:28:05.177605 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 19:28:05.177605 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 19:28:05.177605 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 19:28:05.177605 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 19:28:05.177605 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 19:28:05.177605 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:28:05.180781 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 620/1226] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:28:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:28:05.181678 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e701b0f-a269-4b5b-aa6a-0c2ba86b841d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:05.233339 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:05.293510 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e701b0f-a269-4b5b-aa6a-0c2ba86b841d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:05.336583 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65cc0a1-76df-4871-b151-9982c1a9832a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:05.336897 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cc0a1-76df-4871-b151-9982c1a9832a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:05.337095 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cc0a1-76df-4871-b151-9982c1a9832a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:05.337371 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cc0a1-76df-4871-b151-9982c1a9832a None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:05.338267 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65cc0a1-76df-4871-b151-9982c1a9832a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:05.338580 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65cc0a1-76df-4871-b151-9982c1a9832a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:05.340125 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65cc0a1-76df-4871-b151-9982c1a9832a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:05.359029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 607/1227] 10.222.0.15 () {56 vars in 1143 bytes} [Wed Jun 15 19:28:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:28:05.364713 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361fdb98-141e-4fea-9280-75f8da76f933 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:05.477301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361fdb98-141e-4fea-9280-75f8da76f933 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:05.511499 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6545c-239e-4e6d-8049-dc7d5b50a1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:05.512285 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6545c-239e-4e6d-8049-dc7d5b50a1e7 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:05.512565 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6545c-239e-4e6d-8049-dc7d5b50a1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:05.512835 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6545c-239e-4e6d-8049-dc7d5b50a1e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:05.526853 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f6545c-239e-4e6d-8049-dc7d5b50a1e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:05.528802 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f6545c-239e-4e6d-8049-dc7d5b50a1e7 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:05.550791 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-23f6545c-239e-4e6d-8049-dc7d5b50a1e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:28:05.550791 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:28:05.550791 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:28:05.550791 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:28:05.550791 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:28:05.550791 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:28:05.550791 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:28:05.550791 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:28:05.550791 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:28:05.550791 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:28:05.550791 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:28:05.550791 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:28:05.550791 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:28:05.550791 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:28:05.550791 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:28:05.551908 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:28:05.551908 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:28:05.551908 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:28:05.551908 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:28:05.551908 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:28:05.551908 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:28:05.551908 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:28:05.551908 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:28:05.551908 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:28:05.551908 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:28:05.551908 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:28:05.551908 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:28:05.551908 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:28:05.551908 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:28:05.551908 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:28:05.551908 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:28:05.551908 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:28:05.552852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:28:05.552852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:28:05.552852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:28:05.552852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:28:05.552852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 621/1228] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:28:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:28:05.558199 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65cc0a1-76df-4871-b151-9982c1a9832a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:05.664094 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65cc0a1-76df-4871-b151-9982c1a9832a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:05.666601 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:05.703426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba097a24-7445-4d31-ba48-ec6d23d8cc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:05.704087 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba097a24-7445-4d31-ba48-ec6d23d8cc5b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:05.704412 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba097a24-7445-4d31-ba48-ec6d23d8cc5b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:05.704687 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba097a24-7445-4d31-ba48-ec6d23d8cc5b None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:05.705459 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba097a24-7445-4d31-ba48-ec6d23d8cc5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:05.706333 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba097a24-7445-4d31-ba48-ec6d23d8cc5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:05.707732 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba097a24-7445-4d31-ba48-ec6d23d8cc5b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:05.721822 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 608/1229] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:28:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:28:05.727911 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6545c-239e-4e6d-8049-dc7d5b50a1e7 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:05.739767 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:05.854077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6545c-239e-4e6d-8049-dc7d5b50a1e7 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:05.897539 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0945253c-7480-4517-be4f-e302b465f776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:05.898365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0945253c-7480-4517-be4f-e302b465f776 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:05.898649 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0945253c-7480-4517-be4f-e302b465f776 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:05.898922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0945253c-7480-4517-be4f-e302b465f776 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:05.905844 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0945253c-7480-4517-be4f-e302b465f776 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:05.907319 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0945253c-7480-4517-be4f-e302b465f776 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:05.922051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-0945253c-7480-4517-be4f-e302b465f776 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:28:05.922051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:28:05.922051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:28:05.922051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:28:05.922051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:28:05.922051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:28:05.922051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:28:05.922051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:28:05.922051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:28:05.922051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:28:05.922051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:28:05.922051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:28:05.922051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 19:28:05.922051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 19:28:05.922051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 19:28:05.923743 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 19:28:05.923743 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:28:05.923743 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:28:05.923743 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:28:05.923743 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:28:05.923743 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:28:05.923743 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:28:05.923743 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:28:05.923743 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:28:05.923743 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:28:05.923743 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:28:05.923743 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:28:05.923743 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:28:05.923743 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:28:05.923743 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:28:05.923743 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:28:05.923743 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:28:05.924979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 19:28:05.924979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 19:28:05.924979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:28:05.924979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:28:05.924979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:28:05.924979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:28:05.924979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:28:05.924979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:28:05.924979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 622/1230] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:28:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:28:05.929227 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba097a24-7445-4d31-ba48-ec6d23d8cc5b None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:06.044529 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba097a24-7445-4d31-ba48-ec6d23d8cc5b None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:06.083919 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac43e607-d2b7-49cb-b306-f9ae95a4a8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:06.084622 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43e607-d2b7-49cb-b306-f9ae95a4a8d6 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:06.084911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43e607-d2b7-49cb-b306-f9ae95a4a8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:06.085192 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43e607-d2b7-49cb-b306-f9ae95a4a8d6 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:06.086003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac43e607-d2b7-49cb-b306-f9ae95a4a8d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:06.086358 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac43e607-d2b7-49cb-b306-f9ae95a4a8d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:06.087807 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac43e607-d2b7-49cb-b306-f9ae95a4a8d6 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:06.105776 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 609/1231] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jun 15 19:28:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:28:06.112442 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0945253c-7480-4517-be4f-e302b465f776 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:06.185898 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:06.223135 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0945253c-7480-4517-be4f-e302b465f776 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:06.258951 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77383d-aafd-4ada-8977-c1b319019a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:06.259833 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77383d-aafd-4ada-8977-c1b319019a81 None admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:06.260114 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77383d-aafd-4ada-8977-c1b319019a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:06.260394 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77383d-aafd-4ada-8977-c1b319019a81 None admin] PATH_INFO: `/v3/projects/7fb8cc0eb560465e972ffd45867ee73b/users/cb0ec2ef6c4748e181f1963e7dfe30b2/roles/a8df9714563b435e9fce55a04e74ae5e` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:06.277457 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a77383d-aafd-4ada-8977-c1b319019a81 None admin] RBAC: Authorizing `identity:create_grant(project_id=7fb8cc0eb560465e972ffd45867ee73b, user_id=cb0ec2ef6c4748e181f1963e7dfe30b2, role_id=a8df9714563b435e9fce55a04e74ae5e)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:06.278935 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a77383d-aafd-4ada-8977-c1b319019a81 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:06.292002 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 623/1232] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jun 15 19:28:06 2022] PUT /identity/v3/projects/7fb8cc0eb560465e972ffd45867ee73b/users/cb0ec2ef6c4748e181f1963e7dfe30b2/roles/a8df9714563b435e9fce55a04e74ae5e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:28:08.570594 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fbf1ba-8753-42a5-916c-9ed1ca406eae None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:08.571141 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fbf1ba-8753-42a5-916c-9ed1ca406eae None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:08.571141 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fbf1ba-8753-42a5-916c-9ed1ca406eae None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:08.573024 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 610/1233] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:28:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:28:08.578674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f82e9-8c5f-4972-a043-33d6c98d48f0 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:08.579189 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f82e9-8c5f-4972-a043-33d6c98d48f0 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:08.579448 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f82e9-8c5f-4972-a043-33d6c98d48f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:08.597546 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-730f82e9-8c5f-4972-a043-33d6c98d48f0 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:28:08.598682 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-730f82e9-8c5f-4972-a043-33d6c98d48f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:28:08.676810 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 624/1234] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:28:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:28:08.700226 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fbf1ba-8753-42a5-916c-9ed1ca406eae None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:08.726549 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:08.821391 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fbf1ba-8753-42a5-916c-9ed1ca406eae None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:08.860883 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7964ff44-5f98-484f-8ede-b7f2d1988f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:08.862202 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7964ff44-5f98-484f-8ede-b7f2d1988f53 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:08.862494 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7964ff44-5f98-484f-8ede-b7f2d1988f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:08.862771 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7964ff44-5f98-484f-8ede-b7f2d1988f53 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:08.875383 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7964ff44-5f98-484f-8ede-b7f2d1988f53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:28:08.883989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-7964ff44-5f98-484f-8ede-b7f2d1988f53 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:28:08.883989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:28:08.883989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:28:08.883989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:28:08.883989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:28:08.883989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:28:08.883989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:28:08.883989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:28:08.883989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:28:08.883989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:28:08.883989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:28:08.883989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:28:08.883989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:28:08.883989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:28:08.883989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:28:08.883989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:28:08.885362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:28:08.885362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:28:08.885362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:28:08.885362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:28:08.885362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:28:08.885362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:28:08.885362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:28:08.885362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:28:08.885362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:28:08.885362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:28:08.885362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:28:08.885362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:28:08.885362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:28:08.885362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:28:08.885362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:28:08.885362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:28:08.886594 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:28:08.886594 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:28:08.886594 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:28:08.886594 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:28:08.886594 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:28:08.886594 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:28:08.886594 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:28:08.886594 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:28:08.886594 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:28:08.886594 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:28:08.886594 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 19:28:08.886594 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:28:08.894149 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 611/1235] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Jun 15 19:28:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:28:08.894309 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730f82e9-8c5f-4972-a043-33d6c98d48f0 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:08.941651 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:08.999081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730f82e9-8c5f-4972-a043-33d6c98d48f0 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:09.038198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b495d41-43f5-459f-93ad-009511e6c949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:09.039215 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b495d41-43f5-459f-93ad-009511e6c949 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:09.039532 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b495d41-43f5-459f-93ad-009511e6c949 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:09.039777 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b495d41-43f5-459f-93ad-009511e6c949 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:09.040737 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b495d41-43f5-459f-93ad-009511e6c949 None admin] RBAC: Adding query filter params (name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:09.041042 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b495d41-43f5-459f-93ad-009511e6c949 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:09.042581 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b495d41-43f5-459f-93ad-009511e6c949 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:09.052046 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 625/1236] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Jun 15 19:28:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:28:09.057896 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7964ff44-5f98-484f-8ede-b7f2d1988f53 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:09.157425 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:09.176468 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7964ff44-5f98-484f-8ede-b7f2d1988f53 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:09.214853 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaff7f9-5530-4280-9e4f-8e0e337c2438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:09.215895 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaff7f9-5530-4280-9e4f-8e0e337c2438 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:09.216287 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaff7f9-5530-4280-9e4f-8e0e337c2438 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:09.216683 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaff7f9-5530-4280-9e4f-8e0e337c2438 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:09.230027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaff7f9-5530-4280-9e4f-8e0e337c2438 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:09.232336 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaff7f9-5530-4280-9e4f-8e0e337c2438 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:09.253133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-fcaff7f9-5530-4280-9e4f-8e0e337c2438 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:28:09.253133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:28:09.253133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:28:09.253133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:28:09.253133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:28:09.253133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:28:09.253133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:28:09.253133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:28:09.253133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:28:09.253133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:28:09.253133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:28:09.253133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:28:09.253133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:28:09.253133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:28:09.253133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:28:09.255051 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:28:09.255051 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:28:09.255051 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:28:09.255051 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:28:09.255051 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:28:09.255051 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:28:09.255051 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:28:09.255051 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:28:09.255051 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:28:09.255051 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:28:09.255051 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:28:09.255051 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:28:09.255051 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:28:09.255051 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:28:09.255051 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:28:09.255051 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:28:09.255051 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:28:09.256157 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:28:09.256157 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:28:09.256157 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:28:09.256157 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:28:09.256157 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 612/1237] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:28:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 19:28:09.260691 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b495d41-43f5-459f-93ad-009511e6c949 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:09.367314 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b495d41-43f5-459f-93ad-009511e6c949 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:09.376870 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:09.404184 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63e5d9-3c0e-4100-b575-19feeb8f90c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:09.404990 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63e5d9-3c0e-4100-b575-19feeb8f90c9 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:09.405312 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63e5d9-3c0e-4100-b575-19feeb8f90c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:09.405591 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63e5d9-3c0e-4100-b575-19feeb8f90c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:09.406406 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf63e5d9-3c0e-4100-b575-19feeb8f90c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:09.406718 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf63e5d9-3c0e-4100-b575-19feeb8f90c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:09.408202 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf63e5d9-3c0e-4100-b575-19feeb8f90c9 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:09.421126 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 626/1238] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:28:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:28:09.427387 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaff7f9-5530-4280-9e4f-8e0e337c2438 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:09.535668 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaff7f9-5530-4280-9e4f-8e0e337c2438 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:09.572928 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff5391-0b77-4f39-8526-df9c05bde3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:09.573773 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff5391-0b77-4f39-8526-df9c05bde3b9 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:09.574146 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff5391-0b77-4f39-8526-df9c05bde3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:09.574476 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff5391-0b77-4f39-8526-df9c05bde3b9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:09.577782 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:09.584174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ff5391-0b77-4f39-8526-df9c05bde3b9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:09.586919 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ff5391-0b77-4f39-8526-df9c05bde3b9 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:09.602165 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-e2ff5391-0b77-4f39-8526-df9c05bde3b9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 19:28:09.602165 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:28:09.602165 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:28:09.602165 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:28:09.602165 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:28:09.602165 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:28:09.602165 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:28:09.602165 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:28:09.602165 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:28:09.602165 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:28:09.602165 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:28:09.602165 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:28:09.602165 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 19:28:09.602165 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 19:28:09.602165 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 19:28:09.602165 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 19:28:09.604721 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:28:09.604721 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:28:09.604721 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:28:09.604721 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:28:09.604721 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:28:09.604721 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:28:09.604721 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:28:09.604721 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:28:09.604721 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:28:09.604721 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:28:09.604721 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:28:09.604721 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:28:09.604721 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:28:09.604721 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:28:09.604721 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:28:09.604721 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:28:09.604721 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:28:09.604721 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:28:09.605738 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:28:09.605738 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:28:09.605738 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 19:28:09.605738 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 19:28:09.605738 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 19:28:09.605738 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 19:28:09.605738 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 19:28:09.605738 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 19:28:09.605738 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 19:28:09.605738 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:28:09.605738 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 613/1239] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:28:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:28:09.609712 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63e5d9-3c0e-4100-b575-19feeb8f90c9 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:09.724524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63e5d9-3c0e-4100-b575-19feeb8f90c9 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:09.765204 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940558b8-cfc0-4811-987f-8a6237129bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:09.765936 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940558b8-cfc0-4811-987f-8a6237129bdc None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:09.766246 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940558b8-cfc0-4811-987f-8a6237129bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:09.766518 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940558b8-cfc0-4811-987f-8a6237129bdc None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:09.767347 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940558b8-cfc0-4811-987f-8a6237129bdc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:09.767655 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940558b8-cfc0-4811-987f-8a6237129bdc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:09.769649 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940558b8-cfc0-4811-987f-8a6237129bdc None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:09.791560 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 627/1240] 10.222.0.15 () {56 vars in 1143 bytes} [Wed Jun 15 19:28:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:28:09.797889 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff5391-0b77-4f39-8526-df9c05bde3b9 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:09.910586 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff5391-0b77-4f39-8526-df9c05bde3b9 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:09.948316 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b0124-1d54-46db-89c7-03c9c4244d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:09.948965 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b0124-1d54-46db-89c7-03c9c4244d9d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:09.949265 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b0124-1d54-46db-89c7-03c9c4244d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:09.949627 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b0124-1d54-46db-89c7-03c9c4244d9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:10.022618 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:10.030694 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075b0124-1d54-46db-89c7-03c9c4244d9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:10.032998 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075b0124-1d54-46db-89c7-03c9c4244d9d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:10.151543 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 614/1241] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:28:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 19:28:10.157568 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940558b8-cfc0-4811-987f-8a6237129bdc None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:10.189134 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:10.287056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940558b8-cfc0-4811-987f-8a6237129bdc None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:10.323507 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324935d-51ff-46cb-8613-e5eec1243107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:10.324460 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324935d-51ff-46cb-8613-e5eec1243107 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:10.324749 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324935d-51ff-46cb-8613-e5eec1243107 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:10.325051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324935d-51ff-46cb-8613-e5eec1243107 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:10.341351 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8324935d-51ff-46cb-8613-e5eec1243107 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:10.342460 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8324935d-51ff-46cb-8613-e5eec1243107 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:10.361642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-8324935d-51ff-46cb-8613-e5eec1243107 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:28:10.361642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:28:10.361642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:28:10.361642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:28:10.361642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:28:10.361642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:28:10.361642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:28:10.361642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:28:10.361642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:28:10.361642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:28:10.361642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:28:10.361642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:28:10.361642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:28:10.361642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:28:10.361642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:28:10.362852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:28:10.362852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:28:10.362852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:28:10.362852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:28:10.362852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:28:10.362852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:28:10.362852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:28:10.362852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:28:10.362852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:28:10.362852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:28:10.362852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:28:10.362852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:28:10.362852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:28:10.362852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:28:10.362852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:28:10.362852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:28:10.362852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:28:10.363931 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:28:10.363931 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:28:10.363931 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:28:10.363931 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:28:10.363931 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 628/1242] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:28:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:28:10.368567 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b0124-1d54-46db-89c7-03c9c4244d9d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:10.471675 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:10.479644 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b0124-1d54-46db-89c7-03c9c4244d9d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:10.527748 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80f219-2087-4c34-b4b5-ccb4bfdf2c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:10.528632 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80f219-2087-4c34-b4b5-ccb4bfdf2c01 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:10.529034 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80f219-2087-4c34-b4b5-ccb4bfdf2c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:10.529434 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80f219-2087-4c34-b4b5-ccb4bfdf2c01 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:10.530465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f80f219-2087-4c34-b4b5-ccb4bfdf2c01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:10.530973 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f80f219-2087-4c34-b4b5-ccb4bfdf2c01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:10.533038 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f80f219-2087-4c34-b4b5-ccb4bfdf2c01 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:10.550022 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 615/1243] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:28:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:28:10.556834 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324935d-51ff-46cb-8613-e5eec1243107 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:10.644279 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:10.666865 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324935d-51ff-46cb-8613-e5eec1243107 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:10.705974 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac53ff-3573-4f4c-8fab-f826cdbbb052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:10.706800 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac53ff-3573-4f4c-8fab-f826cdbbb052 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:10.707082 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac53ff-3573-4f4c-8fab-f826cdbbb052 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:10.707355 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac53ff-3573-4f4c-8fab-f826cdbbb052 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:10.714662 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dac53ff-3573-4f4c-8fab-f826cdbbb052 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:10.716400 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dac53ff-3573-4f4c-8fab-f826cdbbb052 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:10.731269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-4dac53ff-3573-4f4c-8fab-f826cdbbb052 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:28:10.731269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:28:10.731269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:28:10.731269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:28:10.731269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:28:10.731269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:28:10.731269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:28:10.731269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:28:10.731269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:28:10.731269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:28:10.731269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:28:10.731269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:28:10.731269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 19:28:10.731269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 19:28:10.731269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 19:28:10.732379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 19:28:10.732379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:28:10.732379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:28:10.732379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:28:10.732379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:28:10.732379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:28:10.732379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:28:10.732379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:28:10.732379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:28:10.732379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:28:10.732379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:28:10.732379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:28:10.732379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:28:10.732379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:28:10.732379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:28:10.732379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:28:10.732379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:28:10.733454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 19:28:10.733454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 19:28:10.733454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:28:10.733454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:28:10.733454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:28:10.733454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:28:10.733454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:28:10.733454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:28:10.733454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 629/1244] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:28:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:28:10.739093 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80f219-2087-4c34-b4b5-ccb4bfdf2c01 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:10.852783 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80f219-2087-4c34-b4b5-ccb4bfdf2c01 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:10.889803 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1a770-3524-40b2-b5cc-c7dc835fd2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:10.890535 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1a770-3524-40b2-b5cc-c7dc835fd2d5 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:10.890901 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1a770-3524-40b2-b5cc-c7dc835fd2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:10.891223 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1a770-3524-40b2-b5cc-c7dc835fd2d5 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:10.892299 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1a770-3524-40b2-b5cc-c7dc835fd2d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:10.892838 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1a770-3524-40b2-b5cc-c7dc835fd2d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:10.894516 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1a770-3524-40b2-b5cc-c7dc835fd2d5 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:10.906663 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 616/1245] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jun 15 19:28:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:28:10.913378 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac53ff-3573-4f4c-8fab-f826cdbbb052 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:11.026406 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac53ff-3573-4f4c-8fab-f826cdbbb052 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:11.064847 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c1d72-8bf9-4028-9820-9c12e4b13aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:11.065549 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c1d72-8bf9-4028-9820-9c12e4b13aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:11.065804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c1d72-8bf9-4028-9820-9c12e4b13aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:11.066077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c1d72-8bf9-4028-9820-9c12e4b13aa5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:11.066935 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808c1d72-8bf9-4028-9820-9c12e4b13aa5 None admin] RBAC: Adding query filter params (role.id=a8df9714563b435e9fce55a04e74ae5e, scope.project.id=7fb8cc0eb560465e972ffd45867ee73b, user.id=cb0ec2ef6c4748e181f1963e7dfe30b2) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:11.067252 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808c1d72-8bf9-4028-9820-9c12e4b13aa5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:11.068689 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808c1d72-8bf9-4028-9820-9c12e4b13aa5 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:11.074728 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 630/1246] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:28:10 2022] GET /identity/v3/role_assignments?user.id=cb0ec2ef6c4748e181f1963e7dfe30b2&scope.project.id=7fb8cc0eb560465e972ffd45867ee73b&role.id=a8df9714563b435e9fce55a04e74ae5e => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:28:13.357022 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a124c2c-9f50-4716-9e95-a14ab7d082ec None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:13.357518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a124c2c-9f50-4716-9e95-a14ab7d082ec None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:13.357609 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a124c2c-9f50-4716-9e95-a14ab7d082ec None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:13.359180 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 617/1247] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:28:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:28:13.365188 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89e9d1-0c64-4ef9-824f-2bf1bc879ec0 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:13.365718 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89e9d1-0c64-4ef9-824f-2bf1bc879ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:13.365836 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89e9d1-0c64-4ef9-824f-2bf1bc879ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:13.371726 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:13.386475 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-5f89e9d1-0c64-4ef9-824f-2bf1bc879ec0 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:28:13.387833 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-5f89e9d1-0c64-4ef9-824f-2bf1bc879ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:28:13.464303 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 631/1248] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:28:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:28:13.489444 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a124c2c-9f50-4716-9e95-a14ab7d082ec None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:13.510024 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:13.608827 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a124c2c-9f50-4716-9e95-a14ab7d082ec None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:13.649423 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1991a443-2664-47eb-b145-e59774d592c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:13.650378 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991a443-2664-47eb-b145-e59774d592c1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:13.650743 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991a443-2664-47eb-b145-e59774d592c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:13.651022 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991a443-2664-47eb-b145-e59774d592c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:13.664049 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1991a443-2664-47eb-b145-e59774d592c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:28:13.675152 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-1991a443-2664-47eb-b145-e59774d592c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:28:13.675152 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:28:13.675152 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:28:13.675152 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:28:13.675152 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:28:13.675152 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:28:13.675152 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:28:13.675152 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:28:13.675152 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:28:13.675152 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:28:13.675152 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:28:13.675152 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:28:13.675152 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:28:13.675152 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:28:13.675152 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:28:13.675152 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:28:13.676254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:28:13.676254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:28:13.676254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:28:13.676254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:28:13.676254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:28:13.676254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:28:13.676254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:28:13.676254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:28:13.676254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:28:13.676254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:28:13.676254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:28:13.676254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:28:13.676254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:28:13.676254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:28:13.676254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:28:13.676254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:28:13.677105 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:28:13.677105 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:28:13.677105 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:28:13.677105 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:28:13.677105 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:28:13.677105 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:28:13.677105 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:28:13.677105 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:28:13.677105 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:28:13.677105 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:28:13.677105 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:28:13.677105 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:28:13.677105 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 618/1249] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jun 15 19:28:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:28:13.683328 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89e9d1-0c64-4ef9-824f-2bf1bc879ec0 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:13.790693 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89e9d1-0c64-4ef9-824f-2bf1bc879ec0 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:13.825129 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:13.831813 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae476fcc-363f-4943-9fec-0d2604a850c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:13.832489 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae476fcc-363f-4943-9fec-0d2604a850c1 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:13.832821 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae476fcc-363f-4943-9fec-0d2604a850c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:13.833148 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae476fcc-363f-4943-9fec-0d2604a850c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:13.834033 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae476fcc-363f-4943-9fec-0d2604a850c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:13.834336 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae476fcc-363f-4943-9fec-0d2604a850c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:13.835858 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae476fcc-363f-4943-9fec-0d2604a850c1 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:13.848086 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 632/1250] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:28:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:28:13.854182 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1991a443-2664-47eb-b145-e59774d592c1 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:13.947940 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:13.975227 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1991a443-2664-47eb-b145-e59774d592c1 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:14.015538 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695672c-6f1f-4cbb-8f23-19b2c6952160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:14.016455 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695672c-6f1f-4cbb-8f23-19b2c6952160 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:14.016746 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695672c-6f1f-4cbb-8f23-19b2c6952160 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:14.017043 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695672c-6f1f-4cbb-8f23-19b2c6952160 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:14.029157 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8695672c-6f1f-4cbb-8f23-19b2c6952160 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:14.030854 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8695672c-6f1f-4cbb-8f23-19b2c6952160 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:14.050465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-8695672c-6f1f-4cbb-8f23-19b2c6952160 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:28:14.050465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:28:14.050465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:28:14.050465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:28:14.050465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:28:14.050465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:28:14.050465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:28:14.050465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:28:14.050465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:28:14.050465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:28:14.050465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:28:14.050465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:28:14.050465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:28:14.050465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:28:14.050465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:28:14.051612 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:28:14.051612 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:28:14.051612 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:28:14.051612 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:28:14.051612 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:28:14.051612 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:28:14.051612 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:28:14.051612 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:28:14.051612 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:28:14.051612 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:28:14.051612 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:28:14.051612 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:28:14.051612 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:28:14.051612 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:28:14.051612 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:28:14.051612 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:28:14.051612 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:28:14.052520 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:28:14.052520 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:28:14.052520 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:28:14.052520 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:28:14.052520 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 619/1251] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:28:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:28:14.058438 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae476fcc-363f-4943-9fec-0d2604a850c1 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:14.171133 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae476fcc-363f-4943-9fec-0d2604a850c1 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:14.212265 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60afc07a-1c0e-46f7-83a9-b18fd83bc92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:14.212858 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60afc07a-1c0e-46f7-83a9-b18fd83bc92f None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:14.213136 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60afc07a-1c0e-46f7-83a9-b18fd83bc92f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:14.213423 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60afc07a-1c0e-46f7-83a9-b18fd83bc92f None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:14.214166 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60afc07a-1c0e-46f7-83a9-b18fd83bc92f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:14.214489 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60afc07a-1c0e-46f7-83a9-b18fd83bc92f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:14.215849 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60afc07a-1c0e-46f7-83a9-b18fd83bc92f None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:14.229272 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 633/1252] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:28:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:28:14.235080 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695672c-6f1f-4cbb-8f23-19b2c6952160 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:14.343393 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695672c-6f1f-4cbb-8f23-19b2c6952160 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:14.372605 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:14.381713 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3996b686-a593-4901-a6e8-ab51a991b287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:14.382552 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996b686-a593-4901-a6e8-ab51a991b287 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:14.382839 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996b686-a593-4901-a6e8-ab51a991b287 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:14.383119 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996b686-a593-4901-a6e8-ab51a991b287 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:14.391278 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3996b686-a593-4901-a6e8-ab51a991b287 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:14.393667 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3996b686-a593-4901-a6e8-ab51a991b287 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:14.408063 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-3996b686-a593-4901-a6e8-ab51a991b287 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 19:28:14.408063 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:28:14.408063 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:28:14.408063 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:28:14.408063 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:28:14.408063 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:28:14.408063 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:28:14.408063 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:28:14.408063 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:28:14.408063 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:28:14.408063 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:28:14.408063 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:28:14.408063 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 19:28:14.408063 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 19:28:14.408063 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 19:28:14.408063 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 19:28:14.409784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:28:14.409784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:28:14.409784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:28:14.409784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:28:14.409784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:28:14.409784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:28:14.409784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:28:14.409784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:28:14.409784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:28:14.409784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:28:14.409784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:28:14.409784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:28:14.409784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:28:14.409784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:28:14.409784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:28:14.409784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:28:14.409784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:28:14.409784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:28:14.410863 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:28:14.410863 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:28:14.410863 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 19:28:14.410863 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 19:28:14.410863 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 19:28:14.410863 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 19:28:14.410863 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 19:28:14.410863 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 19:28:14.410863 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 19:28:14.410863 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:28:14.410863 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 620/1253] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:28:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:28:14.415801 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60afc07a-1c0e-46f7-83a9-b18fd83bc92f None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:14.476842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:14.528021 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60afc07a-1c0e-46f7-83a9-b18fd83bc92f None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:14.569443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25accd-771c-4fac-a1f6-fae7ca2afcee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:14.570130 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25accd-771c-4fac-a1f6-fae7ca2afcee None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:14.570413 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25accd-771c-4fac-a1f6-fae7ca2afcee None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:14.570685 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25accd-771c-4fac-a1f6-fae7ca2afcee None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:14.571516 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d25accd-771c-4fac-a1f6-fae7ca2afcee None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:14.571820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d25accd-771c-4fac-a1f6-fae7ca2afcee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:14.573235 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d25accd-771c-4fac-a1f6-fae7ca2afcee None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:14.591516 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 634/1254] 10.222.0.15 () {56 vars in 1143 bytes} [Wed Jun 15 19:28:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:28:14.597696 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3996b686-a593-4901-a6e8-ab51a991b287 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:14.701227 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3996b686-a593-4901-a6e8-ab51a991b287 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:14.738298 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec3f44d-b554-4b9d-98d4-24fd42525bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:14.738884 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3f44d-b554-4b9d-98d4-24fd42525bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:14.739171 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3f44d-b554-4b9d-98d4-24fd42525bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:14.739448 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3f44d-b554-4b9d-98d4-24fd42525bc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:14.803318 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:14.817426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec3f44d-b554-4b9d-98d4-24fd42525bc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:14.818912 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec3f44d-b554-4b9d-98d4-24fd42525bc2 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:14.932452 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 621/1255] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:28:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 19:28:14.938301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25accd-771c-4fac-a1f6-fae7ca2afcee None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:15.054635 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25accd-771c-4fac-a1f6-fae7ca2afcee None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:15.070825 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:15.095485 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e62a1c-a389-4c5a-a46b-2151e10a6274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:15.096438 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e62a1c-a389-4c5a-a46b-2151e10a6274 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:15.096761 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e62a1c-a389-4c5a-a46b-2151e10a6274 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:15.097149 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e62a1c-a389-4c5a-a46b-2151e10a6274 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:15.110358 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e62a1c-a389-4c5a-a46b-2151e10a6274 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:15.112206 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e62a1c-a389-4c5a-a46b-2151e10a6274 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:15.134614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-f5e62a1c-a389-4c5a-a46b-2151e10a6274 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:28:15.134614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:28:15.134614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:28:15.134614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:28:15.134614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:28:15.134614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:28:15.134614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:28:15.134614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:28:15.134614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:28:15.134614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:28:15.134614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:28:15.134614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:28:15.134614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:28:15.134614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:28:15.134614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:28:15.135692 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:28:15.135692 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:28:15.135692 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:28:15.135692 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:28:15.135692 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:28:15.135692 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:28:15.135692 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:28:15.135692 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:28:15.135692 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:28:15.135692 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:28:15.135692 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:28:15.135692 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:28:15.135692 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:28:15.135692 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:28:15.135692 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:28:15.135692 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:28:15.135692 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:28:15.136667 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:28:15.136667 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:28:15.136667 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:28:15.136667 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:28:15.136667 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 635/1256] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:28:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:28:15.143237 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec3f44d-b554-4b9d-98d4-24fd42525bc2 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:15.243992 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:15.257399 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec3f44d-b554-4b9d-98d4-24fd42525bc2 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:15.293050 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a605e-c733-4822-a77d-3889cc682e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:15.293729 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a605e-c733-4822-a77d-3889cc682e4d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:15.294012 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a605e-c733-4822-a77d-3889cc682e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:15.294316 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a605e-c733-4822-a77d-3889cc682e4d None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:15.295056 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3a605e-c733-4822-a77d-3889cc682e4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:15.295364 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3a605e-c733-4822-a77d-3889cc682e4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:15.296758 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3a605e-c733-4822-a77d-3889cc682e4d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:15.309151 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 622/1257] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:28:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:28:15.315564 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e62a1c-a389-4c5a-a46b-2151e10a6274 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:15.423708 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e62a1c-a389-4c5a-a46b-2151e10a6274 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:15.460801 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f00ac2-dce3-4cad-8c60-a561b3cab961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:15.461608 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f00ac2-dce3-4cad-8c60-a561b3cab961 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:15.461894 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f00ac2-dce3-4cad-8c60-a561b3cab961 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:15.462203 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f00ac2-dce3-4cad-8c60-a561b3cab961 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:15.469174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f00ac2-dce3-4cad-8c60-a561b3cab961 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:15.470728 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f00ac2-dce3-4cad-8c60-a561b3cab961 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:15.484307 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-50f00ac2-dce3-4cad-8c60-a561b3cab961 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:28:15.484307 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:28:15.484307 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:28:15.484307 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:28:15.484307 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:28:15.484307 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:28:15.484307 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:28:15.484307 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:28:15.484307 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:28:15.484307 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:28:15.484307 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:28:15.484307 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:28:15.484307 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 19:28:15.484307 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 19:28:15.484307 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 19:28:15.485368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 19:28:15.485368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:28:15.485368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:28:15.485368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:28:15.485368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:28:15.485368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:28:15.485368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:28:15.485368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:28:15.485368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:28:15.485368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:28:15.485368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:28:15.485368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:28:15.485368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:28:15.485368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:28:15.485368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:28:15.485368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:28:15.485368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:28:15.486878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 19:28:15.486878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 19:28:15.486878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:28:15.486878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:28:15.486878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:28:15.486878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:28:15.486878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:28:15.486878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:28:15.486878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 636/1258] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:28:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:28:15.491875 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a605e-c733-4822-a77d-3889cc682e4d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:15.593634 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a605e-c733-4822-a77d-3889cc682e4d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:15.627967 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea83c5d-65f4-4ad8-a10c-db8f7a71d7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:15.628635 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea83c5d-65f4-4ad8-a10c-db8f7a71d7a8 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:15.628922 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea83c5d-65f4-4ad8-a10c-db8f7a71d7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:15.629199 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea83c5d-65f4-4ad8-a10c-db8f7a71d7a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:15.630306 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea83c5d-65f4-4ad8-a10c-db8f7a71d7a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:15.630619 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea83c5d-65f4-4ad8-a10c-db8f7a71d7a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:15.632065 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea83c5d-65f4-4ad8-a10c-db8f7a71d7a8 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:15.644773 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 623/1259] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jun 15 19:28:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:28:15.650988 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f00ac2-dce3-4cad-8c60-a561b3cab961 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:15.677312 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:15.752879 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f00ac2-dce3-4cad-8c60-a561b3cab961 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:15.785454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9f272d-1564-4ec6-af8a-d5aebc9142c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:15.786114 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f272d-1564-4ec6-af8a-d5aebc9142c7 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:15.786407 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f272d-1564-4ec6-af8a-d5aebc9142c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:15.786684 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f272d-1564-4ec6-af8a-d5aebc9142c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:15.787514 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9f272d-1564-4ec6-af8a-d5aebc9142c7 None admin] RBAC: Adding query filter params (role.id=4f736313c26d4f8db48bbb2f346e50e4, scope.project.id=7fb8cc0eb560465e972ffd45867ee73b, user.id=cb0ec2ef6c4748e181f1963e7dfe30b2) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:15.787831 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9f272d-1564-4ec6-af8a-d5aebc9142c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:15.789253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9f272d-1564-4ec6-af8a-d5aebc9142c7 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:15.794643 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 637/1260] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:28:15 2022] GET /identity/v3/role_assignments?user.id=cb0ec2ef6c4748e181f1963e7dfe30b2&scope.project.id=7fb8cc0eb560465e972ffd45867ee73b&role.id=4f736313c26d4f8db48bbb2f346e50e4 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:28:17.897575 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52e65f-5a35-488d-a36b-5ec691d7fb25 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:17.898131 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52e65f-5a35-488d-a36b-5ec691d7fb25 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:17.898249 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52e65f-5a35-488d-a36b-5ec691d7fb25 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:17.899538 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 624/1261] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:28:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:28:17.904753 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341a79e-c51b-495f-990e-95d6eb5e73bc None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:17.905169 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341a79e-c51b-495f-990e-95d6eb5e73bc None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:17.905320 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341a79e-c51b-495f-990e-95d6eb5e73bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:17.922358 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-6341a79e-c51b-495f-990e-95d6eb5e73bc None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:28:17.923537 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-6341a79e-c51b-495f-990e-95d6eb5e73bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:28:17.987217 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 638/1262] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:28:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:28:18.009046 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a52e65f-5a35-488d-a36b-5ec691d7fb25 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:18.020285 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:18.108956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a52e65f-5a35-488d-a36b-5ec691d7fb25 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:18.145067 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a3b1a-fd3e-4d81-8a02-3c218becad5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:18.145864 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a3b1a-fd3e-4d81-8a02-3c218becad5f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:18.146202 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a3b1a-fd3e-4d81-8a02-3c218becad5f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:18.146458 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a3b1a-fd3e-4d81-8a02-3c218becad5f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:18.156928 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a3b1a-fd3e-4d81-8a02-3c218becad5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:28:18.164597 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-a51a3b1a-fd3e-4d81-8a02-3c218becad5f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:28:18.164597 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:28:18.164597 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:28:18.164597 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:28:18.164597 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:28:18.164597 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:28:18.164597 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:28:18.164597 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:28:18.164597 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:28:18.164597 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:28:18.164597 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:28:18.164597 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:28:18.164597 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:28:18.164597 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:28:18.164597 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:28:18.164597 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:28:18.166499 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:28:18.166499 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:28:18.166499 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:28:18.166499 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:28:18.166499 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:28:18.166499 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:28:18.166499 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:28:18.166499 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:28:18.166499 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:28:18.166499 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:28:18.166499 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:28:18.166499 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:28:18.166499 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:28:18.166499 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:28:18.166499 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:28:18.166499 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:28:18.167554 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:28:18.167554 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:28:18.167554 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:28:18.167554 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:28:18.167554 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:28:18.167554 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:28:18.167554 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:28:18.167554 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:28:18.167554 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:28:18.167554 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:28:18.167554 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:28:18.167554 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:28:18.167554 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 625/1263] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jun 15 19:28:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:28:18.172638 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341a79e-c51b-495f-990e-95d6eb5e73bc None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:18.191283 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:18.271324 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341a79e-c51b-495f-990e-95d6eb5e73bc None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:18.303514 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5778550c-05a5-4155-8c54-8b31528aff33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:18.304170 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5778550c-05a5-4155-8c54-8b31528aff33 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:18.304448 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5778550c-05a5-4155-8c54-8b31528aff33 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:18.304718 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5778550c-05a5-4155-8c54-8b31528aff33 None admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:18.305453 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5778550c-05a5-4155-8c54-8b31528aff33 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:18.305759 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5778550c-05a5-4155-8c54-8b31528aff33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:18.307143 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5778550c-05a5-4155-8c54-8b31528aff33 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:18.315436 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 639/1264] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:28:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:28:18.321054 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a3b1a-fd3e-4d81-8a02-3c218becad5f None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:18.385664 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:18.418113 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a3b1a-fd3e-4d81-8a02-3c218becad5f None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:18.450693 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf8cb0-baf3-4d1a-89c3-a8de79f2775e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:18.451469 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf8cb0-baf3-4d1a-89c3-a8de79f2775e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:18.451751 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf8cb0-baf3-4d1a-89c3-a8de79f2775e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:18.452022 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf8cb0-baf3-4d1a-89c3-a8de79f2775e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:18.462142 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf8cb0-baf3-4d1a-89c3-a8de79f2775e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:18.463584 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf8cb0-baf3-4d1a-89c3-a8de79f2775e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:18.484092 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-dfaf8cb0-baf3-4d1a-89c3-a8de79f2775e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:28:18.484092 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:28:18.484092 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:28:18.484092 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:28:18.484092 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:28:18.484092 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:28:18.484092 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:28:18.484092 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:28:18.484092 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:28:18.484092 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:28:18.484092 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:28:18.484092 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:28:18.484092 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:28:18.484092 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:28:18.484092 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:28:18.485222 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:28:18.485222 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:28:18.485222 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:28:18.485222 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:28:18.485222 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:28:18.485222 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:28:18.485222 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:28:18.485222 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:28:18.485222 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:28:18.485222 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:28:18.485222 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:28:18.485222 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:28:18.485222 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:28:18.485222 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:28:18.485222 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:28:18.485222 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:28:18.485222 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:28:18.486969 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:28:18.486969 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:28:18.486969 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:28:18.486969 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:28:18.486969 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 626/1265] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:28:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:28:18.491757 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5778550c-05a5-4155-8c54-8b31528aff33 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:18.574443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:18.592677 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5778550c-05a5-4155-8c54-8b31528aff33 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:18.625058 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f0e25-a727-4f8e-8fe0-ee92e519396e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:18.625711 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f0e25-a727-4f8e-8fe0-ee92e519396e None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:18.625992 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f0e25-a727-4f8e-8fe0-ee92e519396e None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:18.626298 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f0e25-a727-4f8e-8fe0-ee92e519396e None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:18.627737 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f0e25-a727-4f8e-8fe0-ee92e519396e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:18.628089 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f0e25-a727-4f8e-8fe0-ee92e519396e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:18.629916 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f0e25-a727-4f8e-8fe0-ee92e519396e None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:18.643674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 640/1266] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:28:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:28:18.649460 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf8cb0-baf3-4d1a-89c3-a8de79f2775e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:18.745230 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf8cb0-baf3-4d1a-89c3-a8de79f2775e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:18.764706 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:18.778455 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfaabe6-a5c5-4619-86ff-b469ce96080c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:18.779236 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfaabe6-a5c5-4619-86ff-b469ce96080c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:18.779517 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfaabe6-a5c5-4619-86ff-b469ce96080c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:18.779789 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfaabe6-a5c5-4619-86ff-b469ce96080c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:18.787536 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfaabe6-a5c5-4619-86ff-b469ce96080c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:18.788970 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfaabe6-a5c5-4619-86ff-b469ce96080c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:18.803518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-ebfaabe6-a5c5-4619-86ff-b469ce96080c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 19:28:18.803518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:28:18.803518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:28:18.803518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:28:18.803518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:28:18.803518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:28:18.803518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:28:18.803518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:28:18.803518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:28:18.803518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:28:18.803518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:28:18.803518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:28:18.803518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 19:28:18.803518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 19:28:18.803518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 19:28:18.803518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 19:28:18.804621 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:28:18.804621 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:28:18.804621 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:28:18.804621 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:28:18.804621 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:28:18.804621 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:28:18.804621 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:28:18.804621 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:28:18.804621 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:28:18.804621 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:28:18.804621 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:28:18.804621 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:28:18.804621 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:28:18.804621 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:28:18.804621 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:28:18.804621 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:28:18.804621 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:28:18.804621 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:28:18.805788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:28:18.805788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:28:18.805788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 19:28:18.805788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 19:28:18.805788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 19:28:18.805788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 19:28:18.805788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 19:28:18.805788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 19:28:18.805788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 19:28:18.805788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:28:18.805788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 627/1267] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:28:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:28:18.810437 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f0e25-a727-4f8e-8fe0-ee92e519396e None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:18.905583 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f0e25-a727-4f8e-8fe0-ee92e519396e None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:18.938518 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced7494-4e2d-45cd-970b-a062ca793089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:18.939198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced7494-4e2d-45cd-970b-a062ca793089 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:18.939480 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced7494-4e2d-45cd-970b-a062ca793089 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:18.939755 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced7494-4e2d-45cd-970b-a062ca793089 None admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:18.940641 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ced7494-4e2d-45cd-970b-a062ca793089 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:18.940891 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ced7494-4e2d-45cd-970b-a062ca793089 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:18.942336 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ced7494-4e2d-45cd-970b-a062ca793089 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:18.950680 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:18.959130 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 641/1268] 10.222.0.15 () {56 vars in 1143 bytes} [Wed Jun 15 19:28:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:28:18.964774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfaabe6-a5c5-4619-86ff-b469ce96080c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:19.064009 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfaabe6-a5c5-4619-86ff-b469ce96080c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:19.097005 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc6920-aab5-409b-953b-191dd0e8c70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:19.097794 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc6920-aab5-409b-953b-191dd0e8c70f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:19.098039 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc6920-aab5-409b-953b-191dd0e8c70f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:19.098348 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc6920-aab5-409b-953b-191dd0e8c70f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:19.109095 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc6920-aab5-409b-953b-191dd0e8c70f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:19.110574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc6920-aab5-409b-953b-191dd0e8c70f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:19.128430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-8fbc6920-aab5-409b-953b-191dd0e8c70f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:28:19.128430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:28:19.128430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:28:19.128430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:28:19.128430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:28:19.128430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:28:19.128430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:28:19.128430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:28:19.128430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:28:19.128430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:28:19.128430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:28:19.128430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:28:19.128430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:28:19.128430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:28:19.128430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:28:19.129719 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:28:19.129719 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:28:19.129719 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:28:19.129719 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:28:19.129719 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:28:19.129719 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:28:19.129719 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:28:19.129719 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:28:19.129719 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:28:19.129719 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:28:19.129719 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:28:19.129719 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:28:19.129719 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:28:19.129719 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:28:19.129719 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:28:19.129719 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:28:19.129719 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:28:19.130844 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:28:19.130844 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:28:19.130844 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:28:19.130844 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:28:19.130844 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 628/1269] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:28:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:28:19.135741 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced7494-4e2d-45cd-970b-a062ca793089 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:19.231239 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced7494-4e2d-45cd-970b-a062ca793089 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:19.263398 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428101cc-b73f-4da1-8a9e-046c4faf5ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:19.264038 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428101cc-b73f-4da1-8a9e-046c4faf5ca4 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:19.264321 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428101cc-b73f-4da1-8a9e-046c4faf5ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:19.264596 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428101cc-b73f-4da1-8a9e-046c4faf5ca4 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:19.265301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428101cc-b73f-4da1-8a9e-046c4faf5ca4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:19.265605 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428101cc-b73f-4da1-8a9e-046c4faf5ca4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:19.266977 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428101cc-b73f-4da1-8a9e-046c4faf5ca4 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:19.278167 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 642/1270] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:28:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:28:19.283691 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc6920-aab5-409b-953b-191dd0e8c70f None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:19.315464 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:19.382468 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc6920-aab5-409b-953b-191dd0e8c70f None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:19.415044 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93e8d0-27bd-4b41-88d0-c109452529db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:19.415825 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93e8d0-27bd-4b41-88d0-c109452529db None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:19.416107 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93e8d0-27bd-4b41-88d0-c109452529db None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:19.416383 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93e8d0-27bd-4b41-88d0-c109452529db None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:19.423103 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d93e8d0-27bd-4b41-88d0-c109452529db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:19.425080 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d93e8d0-27bd-4b41-88d0-c109452529db None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:19.439347 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-2d93e8d0-27bd-4b41-88d0-c109452529db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:28:19.439347 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:28:19.439347 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:28:19.439347 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:28:19.439347 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:28:19.439347 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:28:19.439347 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:28:19.439347 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:28:19.439347 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:28:19.439347 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:28:19.439347 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:28:19.439347 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:28:19.439347 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 19:28:19.439347 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 19:28:19.439347 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 19:28:19.440365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 19:28:19.440365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:28:19.440365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:28:19.440365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:28:19.440365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:28:19.440365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:28:19.440365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:28:19.440365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:28:19.440365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:28:19.440365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:28:19.440365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:28:19.440365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:28:19.440365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:28:19.440365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:28:19.440365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:28:19.440365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:28:19.440365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:28:19.441470 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 19:28:19.441470 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 19:28:19.441470 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:28:19.441470 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:28:19.441470 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:28:19.441470 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:28:19.441470 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:28:19.441470 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:28:19.441470 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 629/1271] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:28:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:28:19.446336 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428101cc-b73f-4da1-8a9e-046c4faf5ca4 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:19.502864 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:19.544016 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428101cc-b73f-4da1-8a9e-046c4faf5ca4 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:19.576665 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90645c0b-b7f7-4749-9573-f80b7408ef13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:19.577243 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90645c0b-b7f7-4749-9573-f80b7408ef13 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:19.577524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90645c0b-b7f7-4749-9573-f80b7408ef13 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:19.577801 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90645c0b-b7f7-4749-9573-f80b7408ef13 None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:19.578611 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90645c0b-b7f7-4749-9573-f80b7408ef13 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:19.578920 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90645c0b-b7f7-4749-9573-f80b7408ef13 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:19.580922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90645c0b-b7f7-4749-9573-f80b7408ef13 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:19.592427 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 643/1272] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jun 15 19:28:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:28:19.598305 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93e8d0-27bd-4b41-88d0-c109452529db None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:19.688810 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:19.699623 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93e8d0-27bd-4b41-88d0-c109452529db None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:19.733925 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f412ec9-ccd6-43a0-9703-dd4c42774169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:19.734851 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f412ec9-ccd6-43a0-9703-dd4c42774169 None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:19.735138 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f412ec9-ccd6-43a0-9703-dd4c42774169 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:19.735416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f412ec9-ccd6-43a0-9703-dd4c42774169 None admin] PATH_INFO: `/v3/projects/7fb8cc0eb560465e972ffd45867ee73b/users/cb0ec2ef6c4748e181f1963e7dfe30b2/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:19.737366 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f412ec9-ccd6-43a0-9703-dd4c42774169 None admin] RBAC: Authorizing `identity:create_grant(project_id=7fb8cc0eb560465e972ffd45867ee73b, user_id=cb0ec2ef6c4748e181f1963e7dfe30b2, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:19.738872 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f412ec9-ccd6-43a0-9703-dd4c42774169 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:19.753336 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 630/1273] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Jun 15 19:28:19 2022] PUT /identity/v3/projects/7fb8cc0eb560465e972ffd45867ee73b/users/cb0ec2ef6c4748e181f1963e7dfe30b2/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:28:21.815886 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08393e75-c672-461d-b5c2-103dea41cdba None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:21.816413 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08393e75-c672-461d-b5c2-103dea41cdba None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:21.816532 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08393e75-c672-461d-b5c2-103dea41cdba None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:21.817822 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 644/1274] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:28:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:28:21.822823 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296a477-3170-4c09-9dec-3d3dcba7603c None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:21.823225 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296a477-3170-4c09-9dec-3d3dcba7603c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:21.823376 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296a477-3170-4c09-9dec-3d3dcba7603c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:21.840306 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-e296a477-3170-4c09-9dec-3d3dcba7603c None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:28:21.841473 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e296a477-3170-4c09-9dec-3d3dcba7603c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:28:21.905623 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 631/1275] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:28:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:28:21.928340 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08393e75-c672-461d-b5c2-103dea41cdba None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:22.029050 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08393e75-c672-461d-b5c2-103dea41cdba None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:22.045470 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:22.066323 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62444dcf-7bd6-479c-bed5-fe57a7a0a0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:22.067036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62444dcf-7bd6-479c-bed5-fe57a7a0a0e1 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:22.067316 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62444dcf-7bd6-479c-bed5-fe57a7a0a0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:22.067586 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62444dcf-7bd6-479c-bed5-fe57a7a0a0e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:22.077269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62444dcf-7bd6-479c-bed5-fe57a7a0a0e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 19:28:22.085569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-62444dcf-7bd6-479c-bed5-fe57a7a0a0e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:28:22.085569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:28:22.085569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:28:22.085569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:28:22.085569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:28:22.085569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:28:22.085569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:28:22.085569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:28:22.085569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:28:22.085569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:28:22.085569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:28:22.085569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:28:22.085569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 19:28:22.085569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 19:28:22.085569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 19:28:22.085569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 19:28:22.086813 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 19:28:22.086813 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 19:28:22.086813 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 19:28:22.086813 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 19:28:22.086813 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:28:22.086813 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:28:22.086813 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:28:22.086813 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:28:22.086813 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:28:22.086813 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:28:22.086813 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:28:22.086813 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:28:22.086813 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:28:22.086813 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:28:22.086813 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:28:22.086813 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:28:22.088106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:28:22.088106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:28:22.088106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:28:22.088106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:28:22.088106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 19:28:22.088106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 19:28:22.088106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 19:28:22.088106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 19:28:22.088106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 19:28:22.088106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 19:28:22.088106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 19:28:22.088106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:28:22.088106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 645/1276] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Jun 15 19:28:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:28:22.092743 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296a477-3170-4c09-9dec-3d3dcba7603c None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:22.172949 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:22.191700 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296a477-3170-4c09-9dec-3d3dcba7603c None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:22.223520 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48b3288-8fc2-43f9-a55f-27a8f72376d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:22.224175 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b3288-8fc2-43f9-a55f-27a8f72376d2 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:22.224454 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b3288-8fc2-43f9-a55f-27a8f72376d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:22.224727 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b3288-8fc2-43f9-a55f-27a8f72376d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:22.231171 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48b3288-8fc2-43f9-a55f-27a8f72376d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:22.231171 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48b3288-8fc2-43f9-a55f-27a8f72376d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:22.231171 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48b3288-8fc2-43f9-a55f-27a8f72376d2 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:22.239060 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 632/1277] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Jun 15 19:28:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:28:22.244884 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62444dcf-7bd6-479c-bed5-fe57a7a0a0e1 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:22.343977 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62444dcf-7bd6-479c-bed5-fe57a7a0a0e1 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:22.377977 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370bdada-6322-41fd-b0d9-fc605df0953a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:22.378794 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370bdada-6322-41fd-b0d9-fc605df0953a None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:22.379333 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370bdada-6322-41fd-b0d9-fc605df0953a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:22.379611 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370bdada-6322-41fd-b0d9-fc605df0953a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:22.389307 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370bdada-6322-41fd-b0d9-fc605df0953a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:22.390801 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370bdada-6322-41fd-b0d9-fc605df0953a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:22.407932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-370bdada-6322-41fd-b0d9-fc605df0953a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:28:22.407932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:28:22.407932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:28:22.407932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:28:22.407932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:28:22.407932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:28:22.407932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:28:22.407932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:28:22.407932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:28:22.407932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:28:22.407932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:28:22.407932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:28:22.407932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:28:22.407932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:28:22.407932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:28:22.408972 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:28:22.408972 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:28:22.408972 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:28:22.408972 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:28:22.408972 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:28:22.408972 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:28:22.408972 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:28:22.408972 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:28:22.408972 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:28:22.408972 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:28:22.408972 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:28:22.408972 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:28:22.408972 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:28:22.408972 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:28:22.408972 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:28:22.408972 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:28:22.408972 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:28:22.409942 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:28:22.409942 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:28:22.409942 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:28:22.409942 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:28:22.409942 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 646/1278] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:28:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:28:22.414972 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48b3288-8fc2-43f9-a55f-27a8f72376d2 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:22.511914 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48b3288-8fc2-43f9-a55f-27a8f72376d2 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:22.543890 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b83d23e-cf2c-4567-974b-a311c424179d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:22.544528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83d23e-cf2c-4567-974b-a311c424179d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:22.544806 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83d23e-cf2c-4567-974b-a311c424179d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:22.545076 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83d23e-cf2c-4567-974b-a311c424179d None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:22.545790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b83d23e-cf2c-4567-974b-a311c424179d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:22.546118 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b83d23e-cf2c-4567-974b-a311c424179d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:22.547463 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b83d23e-cf2c-4567-974b-a311c424179d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:22.558584 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 633/1279] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:28:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:28:22.564951 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370bdada-6322-41fd-b0d9-fc605df0953a None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:22.594815 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:22.662739 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370bdada-6322-41fd-b0d9-fc605df0953a None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:22.694872 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c18806-dcbd-40ac-8d1b-365ecd5cafad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:22.695656 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c18806-dcbd-40ac-8d1b-365ecd5cafad None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:22.695936 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c18806-dcbd-40ac-8d1b-365ecd5cafad None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:22.696210 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c18806-dcbd-40ac-8d1b-365ecd5cafad None admin] PATH_INFO: `/v3/users/placement` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:22.702938 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c18806-dcbd-40ac-8d1b-365ecd5cafad None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:22.704355 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c18806-dcbd-40ac-8d1b-365ecd5cafad None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:22.719122 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-b7c18806-dcbd-40ac-8d1b-365ecd5cafad None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 19:28:22.719122 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:28:22.719122 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:28:22.719122 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:28:22.719122 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:28:22.719122 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:28:22.719122 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:28:22.719122 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:28:22.719122 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:28:22.719122 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:28:22.719122 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:28:22.719122 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:28:22.719122 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 19:28:22.719122 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 19:28:22.719122 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 19:28:22.719122 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 19:28:22.720250 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:28:22.720250 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:28:22.720250 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 19:28:22.720250 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:28:22.720250 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 19:28:22.720250 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 19:28:22.720250 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:28:22.720250 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:28:22.720250 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:28:22.720250 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:28:22.720250 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:28:22.720250 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:28:22.720250 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:28:22.720250 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:28:22.720250 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:28:22.720250 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:28:22.720250 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:28:22.720250 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:28:22.721447 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:28:22.721447 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:28:22.721447 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 19:28:22.721447 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 19:28:22.721447 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 19:28:22.721447 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 19:28:22.721447 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 19:28:22.721447 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 19:28:22.721447 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 19:28:22.721447 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:28:22.721447 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 647/1280] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:28:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:28:22.726339 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b83d23e-cf2c-4567-974b-a311c424179d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:22.730041 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:22.821786 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b83d23e-cf2c-4567-974b-a311c424179d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:22.854209 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc35152-f500-4b84-9649-ae00ede8b927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:22.854801 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc35152-f500-4b84-9649-ae00ede8b927 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:22.855077 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc35152-f500-4b84-9649-ae00ede8b927 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:22.855344 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc35152-f500-4b84-9649-ae00ede8b927 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:22.856169 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc35152-f500-4b84-9649-ae00ede8b927 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:22.856472 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc35152-f500-4b84-9649-ae00ede8b927 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:22.857882 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc35152-f500-4b84-9649-ae00ede8b927 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:22.874043 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 634/1281] 10.222.0.15 () {56 vars in 1143 bytes} [Wed Jun 15 19:28:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:28:22.880307 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c18806-dcbd-40ac-8d1b-365ecd5cafad None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:22.966514 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:22.975914 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c18806-dcbd-40ac-8d1b-365ecd5cafad None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:23.009081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ee833-597b-478c-807c-9c216814bdc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:23.009707 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ee833-597b-478c-807c-9c216814bdc8 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:23.009991 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ee833-597b-478c-807c-9c216814bdc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:23.010299 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ee833-597b-478c-807c-9c216814bdc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:23.073909 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ee833-597b-478c-807c-9c216814bdc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:23.075362 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ee833-597b-478c-807c-9c216814bdc8 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:23.163106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:23.173121 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 648/1282] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Jun 15 19:28:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 15 19:28:23.179365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc35152-f500-4b84-9649-ae00ede8b927 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:23.239771 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:23.277763 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc35152-f500-4b84-9649-ae00ede8b927 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:23.311702 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6431a0f1-a689-4e70-9042-bba395fa56b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:23.312486 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431a0f1-a689-4e70-9042-bba395fa56b1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:23.312766 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431a0f1-a689-4e70-9042-bba395fa56b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:23.313040 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431a0f1-a689-4e70-9042-bba395fa56b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:23.322871 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6431a0f1-a689-4e70-9042-bba395fa56b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:23.324311 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6431a0f1-a689-4e70-9042-bba395fa56b1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:23.341477 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-6431a0f1-a689-4e70-9042-bba395fa56b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:28:23.341477 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:28:23.341477 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:28:23.341477 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:28:23.341477 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:28:23.341477 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:28:23.341477 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:28:23.341477 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:28:23.341477 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:28:23.341477 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:28:23.341477 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:28:23.341477 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:28:23.341477 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 19:28:23.341477 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 19:28:23.341477 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 19:28:23.342545 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 19:28:23.342545 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:28:23.342545 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:28:23.342545 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:28:23.342545 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:28:23.342545 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:28:23.342545 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:28:23.342545 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:28:23.342545 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:28:23.342545 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:28:23.342545 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:28:23.342545 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:28:23.342545 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:28:23.342545 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:28:23.342545 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:28:23.342545 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:28:23.342545 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:28:23.343514 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 19:28:23.343514 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 19:28:23.343514 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 19:28:23.343514 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:28:23.343514 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 635/1283] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:28:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:28:23.348805 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ee833-597b-478c-807c-9c216814bdc8 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:23.443200 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ee833-597b-478c-807c-9c216814bdc8 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:23.476280 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d2e11-2dd9-490c-ba16-9bf53a2223c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:23.476922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d2e11-2dd9-490c-ba16-9bf53a2223c6 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:23.477205 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d2e11-2dd9-490c-ba16-9bf53a2223c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:23.477478 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d2e11-2dd9-490c-ba16-9bf53a2223c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:23.478213 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532d2e11-2dd9-490c-ba16-9bf53a2223c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:23.478518 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532d2e11-2dd9-490c-ba16-9bf53a2223c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:23.479862 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532d2e11-2dd9-490c-ba16-9bf53a2223c6 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:23.490625 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 649/1284] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Jun 15 19:28:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:28:23.496021 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6431a0f1-a689-4e70-9042-bba395fa56b1 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:23.593447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6431a0f1-a689-4e70-9042-bba395fa56b1 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:23.606870 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:23.626149 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c29175d-c6b0-4606-8689-f08c5de640bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:23.627455 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29175d-c6b0-4606-8689-f08c5de640bc None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:23.627734 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29175d-c6b0-4606-8689-f08c5de640bc None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:23.628007 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29175d-c6b0-4606-8689-f08c5de640bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:23.634734 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c29175d-c6b0-4606-8689-f08c5de640bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:23.636169 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c29175d-c6b0-4606-8689-f08c5de640bc None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:23.650203 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-8c29175d-c6b0-4606-8689-f08c5de640bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:28:23.650203 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:28:23.650203 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:28:23.650203 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:28:23.650203 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:28:23.650203 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:28:23.650203 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:28:23.650203 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:28:23.650203 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:28:23.650203 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:28:23.650203 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:28:23.650203 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:28:23.650203 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 19:28:23.650203 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 19:28:23.650203 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 19:28:23.651267 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 19:28:23.651267 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:28:23.651267 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:28:23.651267 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:28:23.651267 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:28:23.651267 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:28:23.651267 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:28:23.651267 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:28:23.651267 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 15 19:28:23.651267 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:28:23.651267 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:28:23.651267 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:28:23.651267 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:28:23.651267 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:28:23.651267 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:28:23.651267 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:28:23.651267 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:28:23.652477 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 19:28:23.652477 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 19:28:23.652477 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 19:28:23.652477 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 19:28:23.652477 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 19:28:23.652477 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 19:28:23.652477 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 19:28:23.652477 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 15 19:28:23.652477 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 636/1285] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Jun 15 19:28:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:28:23.656988 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d2e11-2dd9-490c-ba16-9bf53a2223c6 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:23.708263 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:23.756578 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d2e11-2dd9-490c-ba16-9bf53a2223c6 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:23.787897 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e31a38-cdf0-4c56-a688-6853b78f461f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:23.788548 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e31a38-cdf0-4c56-a688-6853b78f461f None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:23.788829 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e31a38-cdf0-4c56-a688-6853b78f461f None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:23.789102 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e31a38-cdf0-4c56-a688-6853b78f461f None admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:23.789877 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e31a38-cdf0-4c56-a688-6853b78f461f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:23.790211 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e31a38-cdf0-4c56-a688-6853b78f461f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:23.791623 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e31a38-cdf0-4c56-a688-6853b78f461f None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:23.803688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 650/1286] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Jun 15 19:28:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:28:23.809573 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c29175d-c6b0-4606-8689-f08c5de640bc None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:23.907104 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c29175d-c6b0-4606-8689-f08c5de640bc None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:23.939680 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9653ef58-c7fb-4b4e-a04b-c36c26bdb98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:23.940311 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9653ef58-c7fb-4b4e-a04b-c36c26bdb98b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:23.940596 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9653ef58-c7fb-4b4e-a04b-c36c26bdb98b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:23.940872 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9653ef58-c7fb-4b4e-a04b-c36c26bdb98b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:23.941690 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9653ef58-c7fb-4b4e-a04b-c36c26bdb98b None admin] RBAC: Adding query filter params (role.id=4f736313c26d4f8db48bbb2f346e50e4, scope.project.id=7fb8cc0eb560465e972ffd45867ee73b, user.id=cb0ec2ef6c4748e181f1963e7dfe30b2) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:23.941999 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9653ef58-c7fb-4b4e-a04b-c36c26bdb98b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:23.943443 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9653ef58-c7fb-4b4e-a04b-c36c26bdb98b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:23.949065 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 637/1287] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Jun 15 19:28:23 2022] GET /identity/v3/role_assignments?user.id=cb0ec2ef6c4748e181f1963e7dfe30b2&scope.project.id=7fb8cc0eb560465e972ffd45867ee73b&role.id=4f736313c26d4f8db48bbb2f346e50e4 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:28:26.045817 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad794f-0263-4fc5-a953-78f2059cf09a None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:26.046369 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad794f-0263-4fc5-a953-78f2059cf09a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:26.046477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad794f-0263-4fc5-a953-78f2059cf09a None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:26.047779 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 651/1288] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:28:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:28:26.053346 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a6935-6fd1-479b-ae5b-ec07978811ce None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:26.053743 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a6935-6fd1-479b-ae5b-ec07978811ce None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:26.053897 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a6935-6fd1-479b-ae5b-ec07978811ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:26.071049 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-353a6935-6fd1-479b-ae5b-ec07978811ce None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:28:26.072230 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-353a6935-6fd1-479b-ae5b-ec07978811ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:28:26.102067 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:26.144420 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 638/1289] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:28:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:28:26.169647 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad794f-0263-4fc5-a953-78f2059cf09a None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:26.267960 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad794f-0263-4fc5-a953-78f2059cf09a None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:26.278161 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:26.301650 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114eb87-b3ac-4586-a70a-6e0f4c41e7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:26.302408 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114eb87-b3ac-4586-a70a-6e0f4c41e7cf None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:26.302692 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114eb87-b3ac-4586-a70a-6e0f4c41e7cf None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:26.302968 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114eb87-b3ac-4586-a70a-6e0f4c41e7cf None admin] PATH_INFO: `/v3/services/placement` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:26.303618 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1114eb87-b3ac-4586-a70a-6e0f4c41e7cf None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:26.304975 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1114eb87-b3ac-4586-a70a-6e0f4c41e7cf None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:26.316604 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-1114eb87-b3ac-4586-a70a-6e0f4c41e7cf None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 19:28:26.316604 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:28:26.316604 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 19:28:26.316604 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 19:28:26.316604 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 19:28:26.316604 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 19:28:26.316604 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 19:28:26.316604 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:28:26.316604 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 19:28:26.316604 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:28:26.316604 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 19:28:26.316604 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:28:26.316604 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:28:26.316604 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:28:26.316604 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:28:26.316604 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:28:26.316604 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:28:26.317848 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:28:26.317848 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:28:26.317848 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:28:26.317848 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:28:26.317848 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:28:26.317848 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:28:26.317848 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 19:28:26.317848 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 19:28:26.317848 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 19:28:26.317848 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 19:28:26.317848 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:28:26.317848 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:28:26.317848 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:28:26.317848 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:28:26.317848 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:28:26.317848 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:28:26.318968 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:28:26.318968 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:28:26.318968 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:28:26.318968 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:28:26.318968 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:28:26.318968 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:28:26.318968 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:28:26.318968 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:28:26.318968 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:28:26.318968 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:28:26.318968 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 19:28:26.318968 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:28:26.318968 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 19:28:26.318968 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:28:26.318968 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 652/1290] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Jun 15 19:28:26 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:28:26.325932 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a6935-6fd1-479b-ae5b-ec07978811ce None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:26.421965 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a6935-6fd1-479b-ae5b-ec07978811ce None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:26.454397 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ffb80-1bd1-4b3f-9fd4-5b91cb924a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:26.455048 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ffb80-1bd1-4b3f-9fd4-5b91cb924a8d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:26.455326 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ffb80-1bd1-4b3f-9fd4-5b91cb924a8d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:26.455599 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ffb80-1bd1-4b3f-9fd4-5b91cb924a8d None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:26.456297 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0ffb80-1bd1-4b3f-9fd4-5b91cb924a8d None admin] RBAC: Adding query filter params (name=placement) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:26.456596 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0ffb80-1bd1-4b3f-9fd4-5b91cb924a8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:26.457953 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0ffb80-1bd1-4b3f-9fd4-5b91cb924a8d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:26.462654 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 639/1291] 10.222.0.15 () {56 vars in 1116 bytes} [Wed Jun 15 19:28:26 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:28:26.468293 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114eb87-b3ac-4586-a70a-6e0f4c41e7cf None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:26.564032 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114eb87-b3ac-4586-a70a-6e0f4c41e7cf None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:26.595748 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f12b270-e5e2-461e-8a49-c6b8a2ba576a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:26.596390 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12b270-e5e2-461e-8a49-c6b8a2ba576a None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:26.596666 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12b270-e5e2-461e-8a49-c6b8a2ba576a None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:26.596933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12b270-e5e2-461e-8a49-c6b8a2ba576a None admin] PATH_INFO: `/v3/services` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:26.597628 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f12b270-e5e2-461e-8a49-c6b8a2ba576a None admin] RBAC: Adding query filter params (type=placement) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:26.597927 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f12b270-e5e2-461e-8a49-c6b8a2ba576a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:26.599304 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f12b270-e5e2-461e-8a49-c6b8a2ba576a None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:26.603554 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 653/1292] 10.222.0.15 () {56 vars in 1116 bytes} [Wed Jun 15 19:28:26 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:28:28.656119 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a842c3c-abaa-45e0-a562-95eaea7706d6 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:28.656622 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a842c3c-abaa-45e0-a562-95eaea7706d6 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:28.656622 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a842c3c-abaa-45e0-a562-95eaea7706d6 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:28.658509 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 640/1293] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:28:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:28:28.663735 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a1cf8-d751-4355-881d-a58b8aa556c3 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:28.664156 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a1cf8-d751-4355-881d-a58b8aa556c3 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:28.664291 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a1cf8-d751-4355-881d-a58b8aa556c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:28.682883 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-1a9a1cf8-d751-4355-881d-a58b8aa556c3 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:28:28.684084 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-1a9a1cf8-d751-4355-881d-a58b8aa556c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:28:28.714875 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:28.752087 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 654/1294] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:28:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:28:28.775041 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a842c3c-abaa-45e0-a562-95eaea7706d6 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:28.813772 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:28.900052 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a842c3c-abaa-45e0-a562-95eaea7706d6 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:28.936019 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc523fab-6b49-44e8-bd0c-20c36d1c28c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:28.936617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc523fab-6b49-44e8-bd0c-20c36d1c28c4 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:28.936901 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc523fab-6b49-44e8-bd0c-20c36d1c28c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:28.937176 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc523fab-6b49-44e8-bd0c-20c36d1c28c4 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:28.937905 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc523fab-6b49-44e8-bd0c-20c36d1c28c4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:28.939339 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc523fab-6b49-44e8-bd0c-20c36d1c28c4 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:28.952015 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 641/1295] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Jun 15 19:28:28 2022] POST /identity/v3/services => generated 246 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:28:31.047129 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5115b65-0f4e-417d-9bf3-d51aca6435f4 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:31.047700 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5115b65-0f4e-417d-9bf3-d51aca6435f4 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:31.047700 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5115b65-0f4e-417d-9bf3-d51aca6435f4 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:31.049067 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 655/1296] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:28:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:28:31.054004 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844651b7-5423-4955-bb8c-5bc9c7c633c8 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:31.054468 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844651b7-5423-4955-bb8c-5bc9c7c633c8 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:31.054568 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844651b7-5423-4955-bb8c-5bc9c7c633c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:31.071345 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-844651b7-5423-4955-bb8c-5bc9c7c633c8 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:28:31.072526 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-844651b7-5423-4955-bb8c-5bc9c7c633c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:28:31.147576 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 642/1297] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:28:31 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:28:31.170943 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5115b65-0f4e-417d-9bf3-d51aca6435f4 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:31.267736 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5115b65-0f4e-417d-9bf3-d51aca6435f4 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:31.300493 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b62e20-07bb-4bc1-8d9d-590b7c8ced76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:31.301209 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b62e20-07bb-4bc1-8d9d-590b7c8ced76 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:31.301489 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b62e20-07bb-4bc1-8d9d-590b7c8ced76 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:31.301760 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b62e20-07bb-4bc1-8d9d-590b7c8ced76 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:31.302433 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b62e20-07bb-4bc1-8d9d-590b7c8ced76 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:31.303784 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b62e20-07bb-4bc1-8d9d-590b7c8ced76 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:31.315405 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-a6b62e20-07bb-4bc1-8d9d-590b7c8ced76 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 19:28:31.315405 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:28:31.315405 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 19:28:31.315405 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 19:28:31.315405 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 19:28:31.315405 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 19:28:31.315405 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 19:28:31.315405 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:28:31.315405 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 19:28:31.315405 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:28:31.315405 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 19:28:31.315405 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:28:31.315405 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:28:31.315405 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:28:31.315405 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:28:31.315405 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:28:31.315405 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:28:31.316551 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:28:31.316551 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:28:31.316551 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:28:31.316551 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:28:31.316551 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:28:31.316551 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:28:31.316551 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 19:28:31.316551 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 19:28:31.316551 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 19:28:31.316551 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 19:28:31.316551 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:28:31.316551 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:28:31.316551 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:28:31.316551 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:28:31.316551 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:28:31.316551 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:28:31.322026 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:28:31.322026 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:28:31.322026 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:28:31.322026 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:28:31.322026 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:28:31.322026 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:28:31.322026 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:28:31.322026 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:28:31.322026 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:28:31.322026 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:28:31.322026 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 19:28:31.322026 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:28:31.322026 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 19:28:31.322026 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:28:31.322026 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 656/1298] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Jun 15 19:28:31 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:28:31.323053 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844651b7-5423-4955-bb8c-5bc9c7c633c8 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:31.340987 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:31.419474 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844651b7-5423-4955-bb8c-5bc9c7c633c8 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:31.451556 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a6f20-3935-4fa2-bd12-46bcef38891b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:31.452209 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a6f20-3935-4fa2-bd12-46bcef38891b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:31.452491 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a6f20-3935-4fa2-bd12-46bcef38891b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:31.452764 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a6f20-3935-4fa2-bd12-46bcef38891b None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:31.453467 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9a6f20-3935-4fa2-bd12-46bcef38891b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:31.453777 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9a6f20-3935-4fa2-bd12-46bcef38891b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:31.455706 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9a6f20-3935-4fa2-bd12-46bcef38891b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:31.460716 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 643/1299] 10.222.0.15 () {56 vars in 1116 bytes} [Wed Jun 15 19:28:31 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:28:31.466611 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b62e20-07bb-4bc1-8d9d-590b7c8ced76 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:31.498698 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:31.565909 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b62e20-07bb-4bc1-8d9d-590b7c8ced76 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:31.598041 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e72165b-7092-419b-b304-c2f407e66c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:31.598707 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72165b-7092-419b-b304-c2f407e66c20 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:31.598990 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72165b-7092-419b-b304-c2f407e66c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:31.599268 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72165b-7092-419b-b304-c2f407e66c20 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:31.600047 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e72165b-7092-419b-b304-c2f407e66c20 None admin] RBAC: Adding query filter params (interface=public, service_id=b9ae464ac5cf40e0bb3a1142bd54a870, region_id=RegionOne) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:31.600357 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e72165b-7092-419b-b304-c2f407e66c20 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:31.601724 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e72165b-7092-419b-b304-c2f407e66c20 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:31.606672 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 657/1300] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Jun 15 19:28:31 2022] GET /identity/v3/endpoints?service_id=b9ae464ac5cf40e0bb3a1142bd54a870&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:28:31.612904 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a6f20-3935-4fa2-bd12-46bcef38891b None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:31.691220 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:31.708722 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a6f20-3935-4fa2-bd12-46bcef38891b None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:31.741890 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70215b2d-c17c-49d0-97bd-3cb3c42a0d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:31.742564 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70215b2d-c17c-49d0-97bd-3cb3c42a0d80 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:31.742850 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70215b2d-c17c-49d0-97bd-3cb3c42a0d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:31.743126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70215b2d-c17c-49d0-97bd-3cb3c42a0d80 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:31.743798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70215b2d-c17c-49d0-97bd-3cb3c42a0d80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:31.745158 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70215b2d-c17c-49d0-97bd-3cb3c42a0d80 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:31.752288 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 644/1301] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Jun 15 19:28:31 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:28:33.865060 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017828f-169e-4c0d-894c-2ea857418974 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:33.865612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017828f-169e-4c0d-894c-2ea857418974 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:33.865612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017828f-169e-4c0d-894c-2ea857418974 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:33.867055 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 658/1302] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:28:33.872180 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3844dcc5-5d45-4cf4-b09d-128564662a21 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:33.872593 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3844dcc5-5d45-4cf4-b09d-128564662a21 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:33.872736 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3844dcc5-5d45-4cf4-b09d-128564662a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:33.889260 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-3844dcc5-5d45-4cf4-b09d-128564662a21 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:28:33.890456 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3844dcc5-5d45-4cf4-b09d-128564662a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:28:33.956488 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 645/1303] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:28:33 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 19:28:33.978974 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4017828f-169e-4c0d-894c-2ea857418974 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:34.069726 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:34.076416 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4017828f-169e-4c0d-894c-2ea857418974 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:34.108376 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dd64ca-2abb-4044-88c0-eee26f4dca19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:34.109160 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd64ca-2abb-4044-88c0-eee26f4dca19 None admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:34.109438 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd64ca-2abb-4044-88c0-eee26f4dca19 None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:34.109709 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd64ca-2abb-4044-88c0-eee26f4dca19 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:34.110421 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dd64ca-2abb-4044-88c0-eee26f4dca19 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:34.111762 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dd64ca-2abb-4044-88c0-eee26f4dca19 None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:34.123303 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application [None req-d3dd64ca-2abb-4044-88c0-eee26f4dca19 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 19:28:34.123303 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:28:34.123303 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 19:28:34.123303 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 19:28:34.123303 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 19:28:34.123303 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 19:28:34.123303 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 19:28:34.123303 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:28:34.123303 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 19:28:34.123303 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:28:34.123303 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 19:28:34.123303 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:28:34.123303 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 19:28:34.123303 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 19:28:34.123303 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 19:28:34.123303 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 19:28:34.123303 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 19:28:34.124445 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 19:28:34.124445 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 19:28:34.124445 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 19:28:34.124445 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 19:28:34.124445 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 19:28:34.124445 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 19:28:34.124445 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 19:28:34.124445 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 19:28:34.124445 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 19:28:34.124445 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 19:28:34.124445 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 19:28:34.124445 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 19:28:34.124445 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 19:28:34.124445 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 19:28:34.124445 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 19:28:34.124445 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 19:28:34.125632 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 19:28:34.125632 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application with Lock( Jun 15 19:28:34.125632 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 19:28:34.125632 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self._enter() Jun 15 19:28:34.125632 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 19:28:34.125632 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 19:28:34.125632 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 19:28:34.125632 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application return self.creator() Jun 15 19:28:34.125632 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 19:28:34.125632 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application created_value = creator( Jun 15 19:28:34.125632 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 19:28:34.125632 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 19:28:34.125632 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 19:28:34.125632 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: ERROR keystone.server.flask.application Jun 15 19:28:34.125632 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 659/1304] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Jun 15 19:28:33 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 19:28:34.131039 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3844dcc5-5d45-4cf4-b09d-128564662a21 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:34.208874 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:34.231362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3844dcc5-5d45-4cf4-b09d-128564662a21 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:34.267298 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c031cbb7-c536-4521-81cd-e1db4edbc87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:34.267967 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031cbb7-c536-4521-81cd-e1db4edbc87f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:34.268250 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031cbb7-c536-4521-81cd-e1db4edbc87f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:34.268526 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031cbb7-c536-4521-81cd-e1db4edbc87f None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:34.269230 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c031cbb7-c536-4521-81cd-e1db4edbc87f None admin] RBAC: Adding query filter params (name=placement) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:28:34.269553 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c031cbb7-c536-4521-81cd-e1db4edbc87f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:34.270939 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c031cbb7-c536-4521-81cd-e1db4edbc87f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:34.275992 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 646/1305] 10.222.0.15 () {56 vars in 1116 bytes} [Wed Jun 15 19:28:34 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:28:34.281684 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dd64ca-2abb-4044-88c0-eee26f4dca19 None admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:28:34.378459 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dd64ca-2abb-4044-88c0-eee26f4dca19 None admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:28:34.410384 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd953358-85cf-416d-8687-9a8c59531c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:28:34.411030 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd953358-85cf-416d-8687-9a8c59531c5c None admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:28:34.411313 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd953358-85cf-416d-8687-9a8c59531c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:28:34.411586 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd953358-85cf-416d-8687-9a8c59531c5c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:28:34.412315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd953358-85cf-416d-8687-9a8c59531c5c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:28:34.413667 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd953358-85cf-416d-8687-9a8c59531c5c None admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:28:34.432627 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 660/1306] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Jun 15 19:28:34 2022] POST /identity/v3/endpoints => generated 331 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:28:44.273431 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=30 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:28:44.428076 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:29:36.780431 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216cebbe-d5fa-4e7a-847c-f48be92964bc None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:29:36.781001 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216cebbe-d5fa-4e7a-847c-f48be92964bc None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:29:36.781001 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216cebbe-d5fa-4e7a-847c-f48be92964bc None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:29:36.782685 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 647/1307] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:29:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:29:36.787717 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970cb1e-9d27-4743-862c-c772a42d6952 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:29:36.788149 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970cb1e-9d27-4743-862c-c772a42d6952 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:29:36.788275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970cb1e-9d27-4743-862c-c772a42d6952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:29:36.952926 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-6970cb1e-9d27-4743-862c-c772a42d6952 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:29:36.954056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-6970cb1e-9d27-4743-862c-c772a42d6952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:29:37.196241 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 661/1308] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:29:37.271016 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c29980-a8f9-4b1a-8a72-7805179ed68e None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:29:37.271549 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c29980-a8f9-4b1a-8a72-7805179ed68e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:29:37.271975 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c29980-a8f9-4b1a-8a72-7805179ed68e None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:29:37.273756 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 648/1309] 10.222.0.15 () {54 vars in 964 bytes} [Wed Jun 15 19:29:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:29:37.280623 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8f30e-d447-4a65-8fba-6d2d4aa10659 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:29:37.281174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8f30e-d447-4a65-8fba-6d2d4aa10659 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:29:37.281509 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8f30e-d447-4a65-8fba-6d2d4aa10659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:29:37.421616 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-fbb8f30e-d447-4a65-8fba-6d2d4aa10659 None None] MFA Rules not processed for user `5c63e0b6ceed4f7cb2e6f9269eb85795`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:29:37.422898 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-fbb8f30e-d447-4a65-8fba-6d2d4aa10659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:29:37.447006 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:29:37.521517 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 662/1310] 10.222.0.15 () {58 vars in 1061 bytes} [Wed Jun 15 19:29:37 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:29:37.532423 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c29980-a8f9-4b1a-8a72-7805179ed68e None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:29:37.535327 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e5c29980-a8f9-4b1a-8a72-7805179ed68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:29:37.668277 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c29980-a8f9-4b1a-8a72-7805179ed68e None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:29:37.757507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9839d85d-be85-422c-afd9-df9b4e610f30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:29:37.758862 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839d85d-be85-422c-afd9-df9b4e610f30 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:29:37.759378 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839d85d-be85-422c-afd9-df9b4e610f30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:29:37.759873 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839d85d-be85-422c-afd9-df9b4e610f30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:29:37.761740 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-9839d85d-be85-422c-afd9-df9b4e610f30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:29:37.822727 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:29:37.880436 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9839d85d-be85-422c-afd9-df9b4e610f30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:29:37.882050 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9839d85d-be85-422c-afd9-df9b4e610f30 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:29:37.902077 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 649/1311] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:29:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:29:40.319443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cdb6de-a037-40a3-a070-2c4c205f66ee None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:29:40.319985 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cdb6de-a037-40a3-a070-2c4c205f66ee None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:29:40.319985 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cdb6de-a037-40a3-a070-2c4c205f66ee None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:29:40.321408 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 663/1312] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:29:40.326686 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb07d8e-f645-4c20-a15e-97327f16cdce None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:29:40.327073 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb07d8e-f645-4c20-a15e-97327f16cdce None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:29:40.327537 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb07d8e-f645-4c20-a15e-97327f16cdce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:29:40.400184 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-0cb07d8e-f645-4c20-a15e-97327f16cdce None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:29:40.401399 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0cb07d8e-f645-4c20-a15e-97327f16cdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:29:40.499903 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 650/1313] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:29:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:29:40.537745 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cdb6de-a037-40a3-a070-2c4c205f66ee None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:29:40.539448 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-a0cdb6de-a037-40a3-a070-2c4c205f66ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:29:40.649582 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cdb6de-a037-40a3-a070-2c4c205f66ee None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:29:40.727103 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:29:40.730294 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801c172-685f-45dc-ad03-935eb86a8865 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:29:40.731179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801c172-685f-45dc-ad03-935eb86a8865 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:29:40.731479 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801c172-685f-45dc-ad03-935eb86a8865 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:29:40.732560 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801c172-685f-45dc-ad03-935eb86a8865 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:29:40.734371 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-3801c172-685f-45dc-ad03-935eb86a8865 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:29:40.844867 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801c172-685f-45dc-ad03-935eb86a8865 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:29:40.846344 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801c172-685f-45dc-ad03-935eb86a8865 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:29:40.869572 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 664/1314] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:29:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:29:43.268100 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5a603-5366-4170-b567-b575f4d6b9e8 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:29:43.268623 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5a603-5366-4170-b567-b575f4d6b9e8 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:29:43.268734 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5a603-5366-4170-b567-b575f4d6b9e8 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:29:43.270025 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 651/1315] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:29:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:29:43.275127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8471ff3-3656-445f-952c-0939c5c7d4c7 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:29:43.275539 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8471ff3-3656-445f-952c-0939c5c7d4c7 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:29:43.275633 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8471ff3-3656-445f-952c-0939c5c7d4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:29:43.339909 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-f8471ff3-3656-445f-952c-0939c5c7d4c7 None None] MFA Rules not processed for user `abb9d50717324682b8fc92f386605834`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:29:43.341139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-f8471ff3-3656-445f-952c-0939c5c7d4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:29:43.391951 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:29:43.435866 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 665/1316] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:29:43 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:29:43.475141 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea5a603-5366-4170-b567-b575f4d6b9e8 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:29:43.476666 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3ea5a603-5366-4170-b567-b575f4d6b9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:29:43.486372 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:29:43.578920 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea5a603-5366-4170-b567-b575f4d6b9e8 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:29:43.649875 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27bac5a-71a9-484e-80d8-b13079650107 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:29:43.650547 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27bac5a-71a9-484e-80d8-b13079650107 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:29:43.650832 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27bac5a-71a9-484e-80d8-b13079650107 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:29:43.651105 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27bac5a-71a9-484e-80d8-b13079650107 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:29:43.652750 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b27bac5a-71a9-484e-80d8-b13079650107 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:29:43.712419 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:29:43.760207 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27bac5a-71a9-484e-80d8-b13079650107 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:29:43.761640 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27bac5a-71a9-484e-80d8-b13079650107 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:29:43.779988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 652/1317] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:29:43 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:29:46.408858 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4406b-3ee1-41ff-8957-9a39c00cf11e None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:29:46.409391 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4406b-3ee1-41ff-8957-9a39c00cf11e None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:29:46.409820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4406b-3ee1-41ff-8957-9a39c00cf11e None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:29:46.411605 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 666/1318] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:29:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:29:46.417908 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74340d53-fe95-44e8-82e0-d745f487b805 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:29:46.418439 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74340d53-fe95-44e8-82e0-d745f487b805 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:29:46.418871 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74340d53-fe95-44e8-82e0-d745f487b805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:29:46.483335 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-74340d53-fe95-44e8-82e0-d745f487b805 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:29:46.484534 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-74340d53-fe95-44e8-82e0-d745f487b805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:29:46.578304 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 653/1319] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:29:46 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:29:46.615612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc4406b-3ee1-41ff-8957-9a39c00cf11e None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:29:46.617132 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-0dc4406b-3ee1-41ff-8957-9a39c00cf11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:29:46.717687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc4406b-3ee1-41ff-8957-9a39c00cf11e None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:29:46.788795 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058c275c-c7ae-4afd-8216-2408cbee362e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:29:46.789430 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c275c-c7ae-4afd-8216-2408cbee362e service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:29:46.789710 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c275c-c7ae-4afd-8216-2408cbee362e service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:29:46.789986 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c275c-c7ae-4afd-8216-2408cbee362e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:29:46.791627 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-058c275c-c7ae-4afd-8216-2408cbee362e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:29:46.799850 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:29:46.899961 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058c275c-c7ae-4afd-8216-2408cbee362e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:29:46.901311 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058c275c-c7ae-4afd-8216-2408cbee362e service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:29:46.920616 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 667/1320] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:29:46 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:29:49.703568 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c89b2-d2bc-4645-9900-11a81ae726bb None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:29:49.704070 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c89b2-d2bc-4645-9900-11a81ae726bb None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:29:49.704070 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c89b2-d2bc-4645-9900-11a81ae726bb None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:29:49.705450 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 654/1321] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:29:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:29:49.710727 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e2744-cd4b-49a8-9fcc-ab6c5431f132 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:29:49.711156 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e2744-cd4b-49a8-9fcc-ab6c5431f132 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:29:49.711281 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e2744-cd4b-49a8-9fcc-ab6c5431f132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:29:49.784901 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-c16e2744-cd4b-49a8-9fcc-ab6c5431f132 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:29:49.786365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-c16e2744-cd4b-49a8-9fcc-ab6c5431f132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:29:49.852994 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:29:49.895957 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 668/1322] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:29:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:29:49.931309 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c89b2-d2bc-4645-9900-11a81ae726bb None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:29:49.932992 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-040c89b2-d2bc-4645-9900-11a81ae726bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:29:49.953048 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:29:50.038747 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c89b2-d2bc-4645-9900-11a81ae726bb None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:29:50.112031 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac9150-90a6-4802-a5ac-a4a9d30ad695 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:29:50.112674 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac9150-90a6-4802-a5ac-a4a9d30ad695 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:29:50.112952 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac9150-90a6-4802-a5ac-a4a9d30ad695 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:29:50.113224 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac9150-90a6-4802-a5ac-a4a9d30ad695 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:29:50.114893 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-28ac9150-90a6-4802-a5ac-a4a9d30ad695 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:29:50.184818 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:29:50.223711 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ac9150-90a6-4802-a5ac-a4a9d30ad695 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:29:50.225165 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ac9150-90a6-4802-a5ac-a4a9d30ad695 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:29:50.245172 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 655/1323] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:29:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:29:52.404690 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f348fb4-2af6-461d-8d02-38aab2d65f10 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:29:52.405210 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f348fb4-2af6-461d-8d02-38aab2d65f10 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:29:52.405329 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f348fb4-2af6-461d-8d02-38aab2d65f10 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:29:52.406660 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 669/1324] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:29:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:29:52.411716 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074234a-4a1e-4b35-bb06-6181bcf58d52 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:29:52.412145 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074234a-4a1e-4b35-bb06-6181bcf58d52 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:29:52.412285 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074234a-4a1e-4b35-bb06-6181bcf58d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:29:52.481260 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-0074234a-4a1e-4b35-bb06-6181bcf58d52 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:29:52.482497 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0074234a-4a1e-4b35-bb06-6181bcf58d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:29:52.578914 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 656/1325] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:29:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:29:52.613496 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f348fb4-2af6-461d-8d02-38aab2d65f10 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:29:52.615047 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-9f348fb4-2af6-461d-8d02-38aab2d65f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:29:52.721755 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f348fb4-2af6-461d-8d02-38aab2d65f10 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:29:52.795365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5bbb5c-e136-4321-b2f0-56a2642dfc2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:29:52.796006 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5bbb5c-e136-4321-b2f0-56a2642dfc2b service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:29:52.796280 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5bbb5c-e136-4321-b2f0-56a2642dfc2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:29:52.796541 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5bbb5c-e136-4321-b2f0-56a2642dfc2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:29:52.798185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-ef5bbb5c-e136-4321-b2f0-56a2642dfc2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:29:52.814981 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:29:52.916568 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5bbb5c-e136-4321-b2f0-56a2642dfc2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:29:52.918007 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5bbb5c-e136-4321-b2f0-56a2642dfc2b service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:29:52.937553 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 670/1326] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:29:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:29:55.128738 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5da67f-e37d-48f9-ae99-fb866dfc7eb0 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:29:55.129323 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5da67f-e37d-48f9-ae99-fb866dfc7eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:29:55.129323 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5da67f-e37d-48f9-ae99-fb866dfc7eb0 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:29:55.130722 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 657/1327] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:29:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:29:55.135913 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75549f55-fcaf-4c31-9df5-e1b255b2fc91 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:29:55.136345 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75549f55-fcaf-4c31-9df5-e1b255b2fc91 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:29:55.136486 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75549f55-fcaf-4c31-9df5-e1b255b2fc91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:29:55.201611 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-75549f55-fcaf-4c31-9df5-e1b255b2fc91 None None] MFA Rules not processed for user `abb9d50717324682b8fc92f386605834`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:29:55.202851 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-75549f55-fcaf-4c31-9df5-e1b255b2fc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:29:55.269695 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:29:55.303017 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 671/1328] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:29:55 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:29:55.343704 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5da67f-e37d-48f9-ae99-fb866dfc7eb0 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:29:55.345247 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ac5da67f-e37d-48f9-ae99-fb866dfc7eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:29:55.375076 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:29:55.465348 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5da67f-e37d-48f9-ae99-fb866dfc7eb0 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:29:55.542749 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df6397-d445-48b5-bbf7-681017b58fa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:29:55.543397 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df6397-d445-48b5-bbf7-681017b58fa4 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:29:55.543704 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df6397-d445-48b5-bbf7-681017b58fa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:29:55.543976 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df6397-d445-48b5-bbf7-681017b58fa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:29:55.545600 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-12df6397-d445-48b5-bbf7-681017b58fa4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:29:55.620636 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:29:55.653432 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df6397-d445-48b5-bbf7-681017b58fa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:29:55.654907 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df6397-d445-48b5-bbf7-681017b58fa4 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:29:55.673278 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 658/1329] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:29:55 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:29:58.166270 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523e4a1-e93d-46ad-906a-608208fcd669 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:29:58.166810 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523e4a1-e93d-46ad-906a-608208fcd669 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:29:58.166810 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523e4a1-e93d-46ad-906a-608208fcd669 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:29:58.168223 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 672/1330] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:29:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:29:58.176572 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d842f14-d379-42eb-bf0b-de816e3d3d37 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:29:58.177005 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d842f14-d379-42eb-bf0b-de816e3d3d37 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:29:58.177226 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d842f14-d379-42eb-bf0b-de816e3d3d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:29:58.248001 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-5d842f14-d379-42eb-bf0b-de816e3d3d37 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:29:58.249228 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-5d842f14-d379-42eb-bf0b-de816e3d3d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:29:58.345236 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 659/1331] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:29:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:29:58.381595 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e523e4a1-e93d-46ad-906a-608208fcd669 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:29:58.383165 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-e523e4a1-e93d-46ad-906a-608208fcd669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:29:58.483941 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e523e4a1-e93d-46ad-906a-608208fcd669 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:29:58.555625 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c92615-5879-4986-b862-076455e2d83d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:29:58.556282 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c92615-5879-4986-b862-076455e2d83d service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:29:58.556563 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c92615-5879-4986-b862-076455e2d83d service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:29:58.556838 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c92615-5879-4986-b862-076455e2d83d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:29:58.558528 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-38c92615-5879-4986-b862-076455e2d83d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:29:58.585134 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:29:58.670396 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c92615-5879-4986-b862-076455e2d83d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:29:58.671842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c92615-5879-4986-b862-076455e2d83d service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:29:58.689979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 673/1332] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:29:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:30:00.865381 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b35b06-d524-49e5-b52e-1d3bf9c1d0b2 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:00.865913 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b35b06-d524-49e5-b52e-1d3bf9c1d0b2 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:00.866047 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b35b06-d524-49e5-b52e-1d3bf9c1d0b2 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:00.867358 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 660/1333] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:30:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:30:00.873183 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b765c494-7294-41ce-ae06-572942f5eec5 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:00.873603 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b765c494-7294-41ce-ae06-572942f5eec5 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:00.873736 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b765c494-7294-41ce-ae06-572942f5eec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:00.937612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-b765c494-7294-41ce-ae06-572942f5eec5 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:30:00.938837 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-b765c494-7294-41ce-ae06-572942f5eec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:30:01.010418 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:30:01.032814 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 674/1334] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:30:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:30:01.088512 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7130a6-58ba-4693-a607-c17267f6d4bb None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:01.088805 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7130a6-58ba-4693-a607-c17267f6d4bb None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:01.089091 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7130a6-58ba-4693-a607-c17267f6d4bb None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:01.091024 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 661/1335] 10.222.0.15 () {54 vars in 964 bytes} [Wed Jun 15 19:30:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:30:01.096958 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db6a13-d1f9-4fb3-b632-6654e38a62fc None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:01.097247 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db6a13-d1f9-4fb3-b632-6654e38a62fc None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:01.097543 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db6a13-d1f9-4fb3-b632-6654e38a62fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:01.171026 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-03db6a13-d1f9-4fb3-b632-6654e38a62fc None None] MFA Rules not processed for user `5c63e0b6ceed4f7cb2e6f9269eb85795`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:30:01.172152 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-03db6a13-d1f9-4fb3-b632-6654e38a62fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:30:01.245605 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 675/1336] 10.222.0.15 () {58 vars in 1061 bytes} [Wed Jun 15 19:30:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:30:01.255340 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7130a6-58ba-4693-a607-c17267f6d4bb None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:30:01.256861 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-4c7130a6-58ba-4693-a607-c17267f6d4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:30:01.296232 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:30:01.360487 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7130a6-58ba-4693-a607-c17267f6d4bb None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:30:01.432483 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c460310-bf51-45a7-bea2-1ab86ca9cfc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:30:01.433052 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c460310-bf51-45a7-bea2-1ab86ca9cfc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:01.433340 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c460310-bf51-45a7-bea2-1ab86ca9cfc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:01.433620 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c460310-bf51-45a7-bea2-1ab86ca9cfc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:01.435877 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-5c460310-bf51-45a7-bea2-1ab86ca9cfc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:30:01.517622 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:30:01.543840 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c460310-bf51-45a7-bea2-1ab86ca9cfc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:30:01.545277 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c460310-bf51-45a7-bea2-1ab86ca9cfc7 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:30:01.562863 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 662/1337] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:30:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:30:04.499361 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a46f90-a3df-4e3a-a926-eb23e0c3ab3b None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:04.499921 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a46f90-a3df-4e3a-a926-eb23e0c3ab3b None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:04.499921 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a46f90-a3df-4e3a-a926-eb23e0c3ab3b None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:04.501561 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 676/1338] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:30:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:30:04.506774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf5740-41a5-4903-b0d2-8889f60783f7 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:04.507192 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf5740-41a5-4903-b0d2-8889f60783f7 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:04.507331 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf5740-41a5-4903-b0d2-8889f60783f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:04.571159 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-4bdf5740-41a5-4903-b0d2-8889f60783f7 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:30:04.572351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-4bdf5740-41a5-4903-b0d2-8889f60783f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:30:04.666062 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 663/1339] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:30:04 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:30:04.701671 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a46f90-a3df-4e3a-a926-eb23e0c3ab3b None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:30:04.703229 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-10a46f90-a3df-4e3a-a926-eb23e0c3ab3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:30:04.777101 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:30:04.811477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a46f90-a3df-4e3a-a926-eb23e0c3ab3b None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:30:04.883791 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb5eba-fd0c-4b9c-8ebb-6384bf827966 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:30:04.884436 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb5eba-fd0c-4b9c-8ebb-6384bf827966 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:04.884714 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb5eba-fd0c-4b9c-8ebb-6384bf827966 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:04.884985 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb5eba-fd0c-4b9c-8ebb-6384bf827966 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:04.886659 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-c0eb5eba-fd0c-4b9c-8ebb-6384bf827966 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:30:04.992493 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eb5eba-fd0c-4b9c-8ebb-6384bf827966 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:30:04.993908 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eb5eba-fd0c-4b9c-8ebb-6384bf827966 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:30:05.012409 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 677/1340] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:30:04 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:30:09.170283 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d619457-d5ae-4dcb-834c-dbcaa7e78c3a None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:09.170850 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d619457-d5ae-4dcb-834c-dbcaa7e78c3a None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:09.170850 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d619457-d5ae-4dcb-834c-dbcaa7e78c3a None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:09.172212 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 664/1341] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:30:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:30:09.177323 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6a69c-1453-4717-ab63-a5bb35efa8b4 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:09.177768 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6a69c-1453-4717-ab63-a5bb35efa8b4 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:09.177913 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6a69c-1453-4717-ab63-a5bb35efa8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:09.184555 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:30:09.249107 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-8ef6a69c-1453-4717-ab63-a5bb35efa8b4 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:30:09.250338 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-8ef6a69c-1453-4717-ab63-a5bb35efa8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:30:09.358082 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 678/1342] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:30:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:30:09.397934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d619457-d5ae-4dcb-834c-dbcaa7e78c3a None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:30:09.400214 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-5d619457-d5ae-4dcb-834c-dbcaa7e78c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:30:09.452493 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:30:09.504147 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d619457-d5ae-4dcb-834c-dbcaa7e78c3a None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:30:09.507744 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0759b17b-917f-46dc-8eca-8eed35e5952b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:30:09.508390 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759b17b-917f-46dc-8eca-8eed35e5952b service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:09.508674 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759b17b-917f-46dc-8eca-8eed35e5952b service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:09.508948 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759b17b-917f-46dc-8eca-8eed35e5952b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:09.518501 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0759b17b-917f-46dc-8eca-8eed35e5952b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:30:09.519936 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0759b17b-917f-46dc-8eca-8eed35e5952b service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:30:09.529677 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 665/1343] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:30:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:30:12.252222 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd11845-5914-414b-a3b0-e33c3b1a9669 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:12.252773 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd11845-5914-414b-a3b0-e33c3b1a9669 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:12.252773 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd11845-5914-414b-a3b0-e33c3b1a9669 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:12.254267 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 679/1344] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:30:12.259972 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc2875-7245-4269-aced-5d83caf44f05 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:12.260517 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc2875-7245-4269-aced-5d83caf44f05 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:12.260517 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc2875-7245-4269-aced-5d83caf44f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:12.290888 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c9bc2875-7245-4269-aced-5d83caf44f05 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:30:12.292120 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c9bc2875-7245-4269-aced-5d83caf44f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:30:12.380459 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 666/1345] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:30:12 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:30:12.418170 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd11845-5914-414b-a3b0-e33c3b1a9669 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:30:12.428820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd11845-5914-414b-a3b0-e33c3b1a9669 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:30:12.432402 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fff172-e39b-4364-a514-ee82b16d374b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:30:12.433065 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fff172-e39b-4364-a514-ee82b16d374b service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:12.433920 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fff172-e39b-4364-a514-ee82b16d374b service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:12.434237 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fff172-e39b-4364-a514-ee82b16d374b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:12.444448 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fff172-e39b-4364-a514-ee82b16d374b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:30:12.445911 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fff172-e39b-4364-a514-ee82b16d374b service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:30:12.456163 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 680/1346] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:30:12 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:30:16.487661 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82fe45e-fe29-4113-b3d5-5098122e3fa9 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:16.488222 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82fe45e-fe29-4113-b3d5-5098122e3fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:16.488222 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82fe45e-fe29-4113-b3d5-5098122e3fa9 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:16.490405 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 667/1347] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:30:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:30:16.495051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479860b-0a46-4612-91f9-1b0621093842 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:16.496147 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479860b-0a46-4612-91f9-1b0621093842 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:16.496147 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479860b-0a46-4612-91f9-1b0621093842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:16.515979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-d479860b-0a46-4612-91f9-1b0621093842 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:30:16.517055 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-d479860b-0a46-4612-91f9-1b0621093842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:30:16.530567 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 681/1348] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:30:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:30:16.568712 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82fe45e-fe29-4113-b3d5-5098122e3fa9 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:30:16.580672 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82fe45e-fe29-4113-b3d5-5098122e3fa9 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:30:16.584650 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cb5c8a-9f78-478c-adee-97d07f10b335 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:30:16.585305 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb5c8a-9f78-478c-adee-97d07f10b335 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:16.585597 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb5c8a-9f78-478c-adee-97d07f10b335 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:16.585872 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb5c8a-9f78-478c-adee-97d07f10b335 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:16.596729 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cb5c8a-9f78-478c-adee-97d07f10b335 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:30:16.598282 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cb5c8a-9f78-478c-adee-97d07f10b335 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:30:16.608936 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 668/1349] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:30:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:30:21.785860 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f05470-3ec7-4e14-b962-7bf2fa2e12b6 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:21.786385 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f05470-3ec7-4e14-b962-7bf2fa2e12b6 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:21.786385 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f05470-3ec7-4e14-b962-7bf2fa2e12b6 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:21.788091 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 682/1350] 10.222.0.15 () {54 vars in 921 bytes} [Wed Jun 15 19:30:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:30:21.800634 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b70d435-cf29-475d-b68c-14c344814434 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:21.801332 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b70d435-cf29-475d-b68c-14c344814434 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:21.801890 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b70d435-cf29-475d-b68c-14c344814434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:21.971561 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-4b70d435-cf29-475d-b68c-14c344814434 None None] MFA Rules not processed for user `cb0ec2ef6c4748e181f1963e7dfe30b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:30:21.973091 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-4b70d435-cf29-475d-b68c-14c344814434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:30:22.036880 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:30:22.132261 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 669/1351] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Jun 15 19:30:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:30:26.509611 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=72 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:30:26.689064 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f8a41-2a10-4e68-91af-77b2f77730cf None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:26.689376 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f8a41-2a10-4e68-91af-77b2f77730cf None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:26.689640 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f8a41-2a10-4e68-91af-77b2f77730cf None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:26.691063 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 683/1352] 10.222.0.15 () {54 vars in 921 bytes} [Wed Jun 15 19:30:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:30:26.696041 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc9c19a-a80a-4248-9eb8-80b211a75cd6 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:26.696491 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc9c19a-a80a-4248-9eb8-80b211a75cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:26.696574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc9c19a-a80a-4248-9eb8-80b211a75cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:26.715635 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-9dc9c19a-a80a-4248-9eb8-80b211a75cd6 None None] MFA Rules not processed for user `cb0ec2ef6c4748e181f1963e7dfe30b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:30:26.716822 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-9dc9c19a-a80a-4248-9eb8-80b211a75cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:30:26.725261 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 670/1353] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Jun 15 19:30:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:30:28.085974 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d9887-4f23-4878-a1bb-b749c4541be2 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:28.086552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d9887-4f23-4878-a1bb-b749c4541be2 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:28.086552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d9887-4f23-4878-a1bb-b749c4541be2 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:28.087989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 684/1354] 10.222.0.15 () {54 vars in 921 bytes} [Wed Jun 15 19:30:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:30:28.093350 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ea4b0-86a9-42d0-856f-475372ede450 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:28.093753 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ea4b0-86a9-42d0-856f-475372ede450 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:28.093904 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ea4b0-86a9-42d0-856f-475372ede450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:28.113161 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-7c6ea4b0-86a9-42d0-856f-475372ede450 None None] MFA Rules not processed for user `cb0ec2ef6c4748e181f1963e7dfe30b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:30:28.114352 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7c6ea4b0-86a9-42d0-856f-475372ede450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:30:28.123462 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 671/1355] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Jun 15 19:30:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:30:31.884784 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e98579-e5ef-48d6-9010-7f2740270ebb None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:31.884784 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e98579-e5ef-48d6-9010-7f2740270ebb None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:31.885912 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e98579-e5ef-48d6-9010-7f2740270ebb None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:31.889456 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 685/1356] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:30:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:30:31.892468 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214b442-e740-4c7f-8227-589bd613c932 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:31.892918 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214b442-e740-4c7f-8227-589bd613c932 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:31.893009 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214b442-e740-4c7f-8227-589bd613c932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:31.911126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-4214b442-e740-4c7f-8227-589bd613c932 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:30:31.912374 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-4214b442-e740-4c7f-8227-589bd613c932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:30:31.920286 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 672/1357] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:30:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:30:31.975866 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c475d-15c6-49d3-bdd4-064e1127357a None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:31.976684 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c475d-15c6-49d3-bdd4-064e1127357a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:31.977547 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c475d-15c6-49d3-bdd4-064e1127357a None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:31.980689 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 686/1358] 10.222.0.15 () {54 vars in 961 bytes} [Wed Jun 15 19:30:31 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:30:31.990607 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32eb42d-59b2-4819-8957-267d96743c5f None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:31.991222 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32eb42d-59b2-4819-8957-267d96743c5f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:31.991553 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32eb42d-59b2-4819-8957-267d96743c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:32.048087 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c32eb42d-59b2-4819-8957-267d96743c5f None None] MFA Rules not processed for user `3e74ac44e5814122a3226ac023f3688a`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:30:32.049885 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c32eb42d-59b2-4819-8957-267d96743c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:30:32.141959 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 673/1359] 10.222.0.15 () {58 vars in 1058 bytes} [Wed Jun 15 19:30:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:30:32.149827 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c475d-15c6-49d3-bdd4-064e1127357a None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:30:32.164211 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c475d-15c6-49d3-bdd4-064e1127357a None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:30:32.170185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a0dcb4-db44-4e69-b607-74185ad38329 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:30:32.170905 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0dcb4-db44-4e69-b607-74185ad38329 service nova] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:32.171240 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0dcb4-db44-4e69-b607-74185ad38329 service nova] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:32.171582 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0dcb4-db44-4e69-b607-74185ad38329 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:32.180237 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a0dcb4-db44-4e69-b607-74185ad38329 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:30:32.181961 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a0dcb4-db44-4e69-b607-74185ad38329 service nova] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:30:32.201315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 687/1360] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:30:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:30:34.546801 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc8f98-7f0c-462c-b339-b202d1cb9984 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:34.547323 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc8f98-7f0c-462c-b339-b202d1cb9984 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:34.547439 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc8f98-7f0c-462c-b339-b202d1cb9984 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:34.548907 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 674/1361] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:30:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:30:34.559615 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41659b2-da0b-4a70-9496-acb2fa865c2e None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:34.560157 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41659b2-da0b-4a70-9496-acb2fa865c2e None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:34.560157 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41659b2-da0b-4a70-9496-acb2fa865c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:34.580192 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-a41659b2-da0b-4a70-9496-acb2fa865c2e None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:30:34.582604 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-a41659b2-da0b-4a70-9496-acb2fa865c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:30:34.592365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 688/1362] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:30:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:30:34.638334 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc8f98-7f0c-462c-b339-b202d1cb9984 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:30:34.654886 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc8f98-7f0c-462c-b339-b202d1cb9984 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:30:34.668331 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8ead7b-6b74-46be-a976-fc10f076a055 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:30:34.668998 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ead7b-6b74-46be-a976-fc10f076a055 service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:34.669283 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ead7b-6b74-46be-a976-fc10f076a055 service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:34.669562 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ead7b-6b74-46be-a976-fc10f076a055 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:34.680376 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8ead7b-6b74-46be-a976-fc10f076a055 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:30:34.682600 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8ead7b-6b74-46be-a976-fc10f076a055 service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:30:34.693067 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 675/1363] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:30:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:30:37.157067 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d2c4d6-d127-469e-a31b-6b0c2bbaada9 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:37.157868 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d2c4d6-d127-469e-a31b-6b0c2bbaada9 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:37.158276 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d2c4d6-d127-469e-a31b-6b0c2bbaada9 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:37.160097 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 689/1364] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:30:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:30:37.165371 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156aa62-dcaa-4a6b-8e48-8e3d8c93d22b None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:37.165805 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156aa62-dcaa-4a6b-8e48-8e3d8c93d22b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:37.166068 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156aa62-dcaa-4a6b-8e48-8e3d8c93d22b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:37.183581 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-f156aa62-dcaa-4a6b-8e48-8e3d8c93d22b None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:30:37.184770 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f156aa62-dcaa-4a6b-8e48-8e3d8c93d22b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:30:37.192557 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 676/1365] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:30:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:30:37.238833 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeaa6e7-6dfc-4d91-bb54-d5f1531ee755 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:37.239119 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeaa6e7-6dfc-4d91-bb54-d5f1531ee755 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:37.239398 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeaa6e7-6dfc-4d91-bb54-d5f1531ee755 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:37.241188 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 690/1366] 10.222.0.15 () {54 vars in 961 bytes} [Wed Jun 15 19:30:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:30:37.246157 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f143a29-4599-4204-be17-a7be5052fc5e None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:37.246456 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f143a29-4599-4204-be17-a7be5052fc5e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:37.246742 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f143a29-4599-4204-be17-a7be5052fc5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:37.265537 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-1f143a29-4599-4204-be17-a7be5052fc5e None None] MFA Rules not processed for user `3e74ac44e5814122a3226ac023f3688a`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:30:37.266763 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1f143a29-4599-4204-be17-a7be5052fc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:30:37.276072 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 677/1367] 10.222.0.15 () {58 vars in 1058 bytes} [Wed Jun 15 19:30:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:30:37.283615 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeaa6e7-6dfc-4d91-bb54-d5f1531ee755 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:30:37.298731 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeaa6e7-6dfc-4d91-bb54-d5f1531ee755 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:30:37.304955 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d0b42-e622-408b-b573-cef1d95371c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:30:37.305853 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d0b42-e622-408b-b573-cef1d95371c9 service nova] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:37.308666 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d0b42-e622-408b-b573-cef1d95371c9 service nova] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:37.309049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d0b42-e622-408b-b573-cef1d95371c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:37.319153 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81d0b42-e622-408b-b573-cef1d95371c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:30:37.320820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81d0b42-e622-408b-b573-cef1d95371c9 service nova] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:30:37.342297 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 691/1368] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:30:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:30:39.645913 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d247da2b-dc9c-49a2-93d5-2dd8adef2343 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:39.646791 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d247da2b-dc9c-49a2-93d5-2dd8adef2343 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:39.647075 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d247da2b-dc9c-49a2-93d5-2dd8adef2343 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:39.648700 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 678/1369] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:30:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:30:39.654206 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626e269-64ba-451c-8be6-dca1db335730 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:39.655123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626e269-64ba-451c-8be6-dca1db335730 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:39.655609 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626e269-64ba-451c-8be6-dca1db335730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:39.676730 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-1626e269-64ba-451c-8be6-dca1db335730 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:30:39.678703 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-1626e269-64ba-451c-8be6-dca1db335730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:30:39.692969 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 692/1370] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:30:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:30:39.746544 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d247da2b-dc9c-49a2-93d5-2dd8adef2343 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:30:39.769488 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d247da2b-dc9c-49a2-93d5-2dd8adef2343 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:30:39.777106 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae482a-93eb-4433-977b-0e6b0045cc4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:30:39.778045 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae482a-93eb-4433-977b-0e6b0045cc4c service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:39.778657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae482a-93eb-4433-977b-0e6b0045cc4c service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:39.779137 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae482a-93eb-4433-977b-0e6b0045cc4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:39.790283 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae482a-93eb-4433-977b-0e6b0045cc4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:30:39.792967 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae482a-93eb-4433-977b-0e6b0045cc4c service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:30:39.819188 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 679/1371] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:30:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:30:40.177193 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8425f17-68d0-490d-8603-e72dc7ca9875 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:40.177505 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8425f17-68d0-490d-8603-e72dc7ca9875 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:40.177825 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8425f17-68d0-490d-8603-e72dc7ca9875 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:40.183202 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 693/1372] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:30:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:30:40.185149 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a810a51-4922-4ac9-89be-d7302b4aa077 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:40.185446 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a810a51-4922-4ac9-89be-d7302b4aa077 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:40.185726 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a810a51-4922-4ac9-89be-d7302b4aa077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:40.208637 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-4a810a51-4922-4ac9-89be-d7302b4aa077 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:30:40.209763 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-4a810a51-4922-4ac9-89be-d7302b4aa077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:30:40.219301 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 680/1373] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:30:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:30:40.547920 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b97051-1815-4368-9763-3dbf81d4f0b7 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:40.547920 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b97051-1815-4368-9763-3dbf81d4f0b7 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:40.547920 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b97051-1815-4368-9763-3dbf81d4f0b7 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:40.547920 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 694/1374] 10.222.0.15 () {54 vars in 963 bytes} [Wed Jun 15 19:30:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:30:40.553948 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4104ad-a9e0-454f-95b8-00daf8ee1a18 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:40.554646 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4104ad-a9e0-454f-95b8-00daf8ee1a18 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:40.555034 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4104ad-a9e0-454f-95b8-00daf8ee1a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:40.563953 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:30:40.617430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-9c4104ad-a9e0-454f-95b8-00daf8ee1a18 None None] MFA Rules not processed for user `ec5882def40a4e0d8a792eecc12440a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:30:40.618869 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-9c4104ad-a9e0-454f-95b8-00daf8ee1a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:30:40.693529 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 681/1375] 10.222.0.15 () {58 vars in 1060 bytes} [Wed Jun 15 19:30:40 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:30:40.701681 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b97051-1815-4368-9763-3dbf81d4f0b7 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:30:40.718431 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b97051-1815-4368-9763-3dbf81d4f0b7 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:30:40.722431 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab99cc7-7670-42ce-9dcf-ddabb295904d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:30:40.723814 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab99cc7-7670-42ce-9dcf-ddabb295904d service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:40.724533 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab99cc7-7670-42ce-9dcf-ddabb295904d service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:40.725038 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab99cc7-7670-42ce-9dcf-ddabb295904d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:40.736024 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab99cc7-7670-42ce-9dcf-ddabb295904d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:30:40.737901 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab99cc7-7670-42ce-9dcf-ddabb295904d service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:30:40.750613 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 695/1376] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:30:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:30:42.040267 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186bd9c4-3cd3-4fb6-a12d-5ca7ef2e428f None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:42.041867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186bd9c4-3cd3-4fb6-a12d-5ca7ef2e428f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:42.042394 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186bd9c4-3cd3-4fb6-a12d-5ca7ef2e428f None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:42.043782 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 682/1377] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:30:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:30:42.049038 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa14de-9481-4ff3-9ee8-4f27a2b43f60 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:42.049449 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa14de-9481-4ff3-9ee8-4f27a2b43f60 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:42.049602 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa14de-9481-4ff3-9ee8-4f27a2b43f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:42.068465 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-b7aa14de-9481-4ff3-9ee8-4f27a2b43f60 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:30:42.069686 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-b7aa14de-9481-4ff3-9ee8-4f27a2b43f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:30:42.077916 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 696/1378] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:30:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:30:42.123191 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186bd9c4-3cd3-4fb6-a12d-5ca7ef2e428f None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:30:42.138135 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186bd9c4-3cd3-4fb6-a12d-5ca7ef2e428f None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:30:42.144197 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59062524-1207-432a-b1d8-87cbd5f2205d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:30:42.145123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59062524-1207-432a-b1d8-87cbd5f2205d service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:42.145628 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59062524-1207-432a-b1d8-87cbd5f2205d service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:42.146132 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59062524-1207-432a-b1d8-87cbd5f2205d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:42.155708 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59062524-1207-432a-b1d8-87cbd5f2205d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:30:42.157385 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59062524-1207-432a-b1d8-87cbd5f2205d service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:30:42.168047 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 683/1379] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:30:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:30:44.386439 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785335e-4c86-4a4a-b3d4-5ede330a9e67 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:44.386953 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785335e-4c86-4a4a-b3d4-5ede330a9e67 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:44.387050 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785335e-4c86-4a4a-b3d4-5ede330a9e67 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:44.388397 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 697/1380] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:30:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 19:30:44.393814 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633e8ba-5f72-4e08-91c1-b032ff10ec3d None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:44.395087 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633e8ba-5f72-4e08-91c1-b032ff10ec3d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:44.395407 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633e8ba-5f72-4e08-91c1-b032ff10ec3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:44.415358 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-0633e8ba-5f72-4e08-91c1-b032ff10ec3d None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:30:44.418984 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0633e8ba-5f72-4e08-91c1-b032ff10ec3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:30:44.430450 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 684/1381] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:30:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:30:44.474899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6785335e-4c86-4a4a-b3d4-5ede330a9e67 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:30:44.494031 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6785335e-4c86-4a4a-b3d4-5ede330a9e67 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:30:44.506406 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd26f12-45c0-4ab8-9efa-d199f209c315 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:30:44.507412 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd26f12-45c0-4ab8-9efa-d199f209c315 service nova] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:44.507881 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd26f12-45c0-4ab8-9efa-d199f209c315 service nova] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:44.510125 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd26f12-45c0-4ab8-9efa-d199f209c315 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:44.532901 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd26f12-45c0-4ab8-9efa-d199f209c315 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:30:44.534653 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd26f12-45c0-4ab8-9efa-d199f209c315 service nova] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:30:44.545496 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 698/1382] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:30:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:30:46.796423 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513dbcf9-e728-4d28-a043-9ea8696e6a13 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:46.796917 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513dbcf9-e728-4d28-a043-9ea8696e6a13 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:46.797018 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513dbcf9-e728-4d28-a043-9ea8696e6a13 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:46.798572 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 685/1383] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:30:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:30:46.811286 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c8b3c-7c94-44dd-a74c-a73e3d9d3802 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:46.813719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c8b3c-7c94-44dd-a74c-a73e3d9d3802 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:46.814902 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c8b3c-7c94-44dd-a74c-a73e3d9d3802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:46.879302 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-be5c8b3c-7c94-44dd-a74c-a73e3d9d3802 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:30:46.880556 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-be5c8b3c-7c94-44dd-a74c-a73e3d9d3802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:30:46.902263 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 699/1384] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:30:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:30:46.932692 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513dbcf9-e728-4d28-a043-9ea8696e6a13 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:30:46.966470 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513dbcf9-e728-4d28-a043-9ea8696e6a13 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:30:46.989219 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7526ae2-9af8-4ce1-a867-b5f7dc85b707 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:30:46.993119 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7526ae2-9af8-4ce1-a867-b5f7dc85b707 service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:46.994867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7526ae2-9af8-4ce1-a867-b5f7dc85b707 service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:46.996660 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7526ae2-9af8-4ce1-a867-b5f7dc85b707 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:47.029381 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7526ae2-9af8-4ce1-a867-b5f7dc85b707 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:30:47.038539 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7526ae2-9af8-4ce1-a867-b5f7dc85b707 service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:30:47.082534 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 686/1385] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:30:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:30:49.300867 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c6ec8-a758-4fe4-bf02-1b0e8fa7ea3e None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:49.301450 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c6ec8-a758-4fe4-bf02-1b0e8fa7ea3e None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:49.301614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c6ec8-a758-4fe4-bf02-1b0e8fa7ea3e None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:49.303886 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 700/1386] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:30:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 19:30:49.309079 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27360cd-3edb-4bf1-8dc9-aa83c1e72190 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:49.309570 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27360cd-3edb-4bf1-8dc9-aa83c1e72190 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:49.309680 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27360cd-3edb-4bf1-8dc9-aa83c1e72190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:49.328948 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c27360cd-3edb-4bf1-8dc9-aa83c1e72190 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:30:49.330299 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c27360cd-3edb-4bf1-8dc9-aa83c1e72190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:30:49.339264 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 687/1387] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:30:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:30:49.387194 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c6ec8-a758-4fe4-bf02-1b0e8fa7ea3e None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:30:49.403949 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c6ec8-a758-4fe4-bf02-1b0e8fa7ea3e None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:30:49.410605 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7199c-a889-42cf-a532-b4f28090d6c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:30:49.411222 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7199c-a889-42cf-a532-b4f28090d6c6 service nova] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:49.411505 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7199c-a889-42cf-a532-b4f28090d6c6 service nova] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:49.411782 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7199c-a889-42cf-a532-b4f28090d6c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:49.420512 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d7199c-a889-42cf-a532-b4f28090d6c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:30:49.422121 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d7199c-a889-42cf-a532-b4f28090d6c6 service nova] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:30:49.432616 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 701/1388] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:30:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:30:51.792180 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5807a5-6499-4634-8253-731605bef109 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:51.793165 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5807a5-6499-4634-8253-731605bef109 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:51.793498 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5807a5-6499-4634-8253-731605bef109 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:51.795010 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 688/1389] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:30:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:30:51.800471 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8335ce9-6d7d-483e-9afe-e02d634587cd None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:51.800919 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8335ce9-6d7d-483e-9afe-e02d634587cd None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:51.801020 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8335ce9-6d7d-483e-9afe-e02d634587cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:51.821651 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-f8335ce9-6d7d-483e-9afe-e02d634587cd None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:30:51.823343 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-f8335ce9-6d7d-483e-9afe-e02d634587cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:30:51.831865 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 702/1390] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:30:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:30:51.881529 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5807a5-6499-4634-8253-731605bef109 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:30:51.900003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5807a5-6499-4634-8253-731605bef109 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:30:51.906184 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731847b-3fea-4960-a84c-0463875bd512 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:30:51.906842 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731847b-3fea-4960-a84c-0463875bd512 service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:51.907140 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731847b-3fea-4960-a84c-0463875bd512 service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:51.907433 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731847b-3fea-4960-a84c-0463875bd512 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:51.916659 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6731847b-3fea-4960-a84c-0463875bd512 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:30:51.918344 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6731847b-3fea-4960-a84c-0463875bd512 service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:30:51.932374 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 689/1391] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:30:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:30:54.107105 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f23d46-7663-48ab-8d9e-c3a3d1d9c440 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:54.107691 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f23d46-7663-48ab-8d9e-c3a3d1d9c440 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:54.107848 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f23d46-7663-48ab-8d9e-c3a3d1d9c440 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:54.109971 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 703/1392] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:30:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:30:54.115349 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3209b16-8f1f-4190-a175-bf1eb63c4ec7 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:54.115940 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3209b16-8f1f-4190-a175-bf1eb63c4ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:54.115940 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3209b16-8f1f-4190-a175-bf1eb63c4ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:54.134364 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-e3209b16-8f1f-4190-a175-bf1eb63c4ec7 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:30:54.136079 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e3209b16-8f1f-4190-a175-bf1eb63c4ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:30:54.144042 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 690/1393] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:30:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:30:54.192021 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f23d46-7663-48ab-8d9e-c3a3d1d9c440 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:30:54.208220 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f23d46-7663-48ab-8d9e-c3a3d1d9c440 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:30:54.213969 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf8947-1881-49b1-aa31-71a6616d46a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:30:54.214653 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf8947-1881-49b1-aa31-71a6616d46a8 service nova] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:54.214953 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf8947-1881-49b1-aa31-71a6616d46a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:54.215264 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf8947-1881-49b1-aa31-71a6616d46a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:54.224439 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bf8947-1881-49b1-aa31-71a6616d46a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:30:54.226066 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bf8947-1881-49b1-aa31-71a6616d46a8 service nova] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:30:54.235824 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 704/1394] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:30:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:30:56.400099 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673e805-3088-4fe8-995f-a57e52d81649 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:56.401226 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673e805-3088-4fe8-995f-a57e52d81649 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:56.401226 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673e805-3088-4fe8-995f-a57e52d81649 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:56.402837 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 691/1395] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:30:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:30:56.407941 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df252679-c72e-4802-9a3f-da4c9f7b5fb2 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:56.408405 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df252679-c72e-4802-9a3f-da4c9f7b5fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:56.408518 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df252679-c72e-4802-9a3f-da4c9f7b5fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:56.427930 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-df252679-c72e-4802-9a3f-da4c9f7b5fb2 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:30:56.429112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-df252679-c72e-4802-9a3f-da4c9f7b5fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:30:56.437941 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 705/1396] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:30:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:30:56.481976 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7673e805-3088-4fe8-995f-a57e52d81649 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:30:56.496417 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7673e805-3088-4fe8-995f-a57e52d81649 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:30:56.501981 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a257a-1c3f-46b7-ad74-442ce46d620a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:30:56.502649 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a257a-1c3f-46b7-ad74-442ce46d620a service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:56.502932 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a257a-1c3f-46b7-ad74-442ce46d620a service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:56.503208 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a257a-1c3f-46b7-ad74-442ce46d620a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:56.511492 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a257a-1c3f-46b7-ad74-442ce46d620a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:30:56.513062 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a257a-1c3f-46b7-ad74-442ce46d620a service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:30:56.522832 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 692/1397] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:30:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:30:56.839737 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a2475-ba47-4691-bd8c-876f942d414a None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:56.840028 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a2475-ba47-4691-bd8c-876f942d414a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:56.840306 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a2475-ba47-4691-bd8c-876f942d414a None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:56.841698 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 706/1398] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:30:56.846821 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394e23d-e6f8-4cad-b8d8-ae850f5614ac None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:56.847109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394e23d-e6f8-4cad-b8d8-ae850f5614ac None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:56.847412 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394e23d-e6f8-4cad-b8d8-ae850f5614ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:56.865308 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-0394e23d-e6f8-4cad-b8d8-ae850f5614ac None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:30:56.866546 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0394e23d-e6f8-4cad-b8d8-ae850f5614ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:30:56.874850 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 693/1399] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:30:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:30:56.929265 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb5408-529c-4c6c-919e-810aaed3c626 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:56.929551 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb5408-529c-4c6c-919e-810aaed3c626 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:56.929826 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb5408-529c-4c6c-919e-810aaed3c626 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:56.931264 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 707/1400] 10.222.0.15 () {54 vars in 962 bytes} [Wed Jun 15 19:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:30:56.936904 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682951d-ec71-40e4-a370-535487b2a4cc None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:56.937315 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682951d-ec71-40e4-a370-535487b2a4cc None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:56.937591 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682951d-ec71-40e4-a370-535487b2a4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:56.985377 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-2682951d-ec71-40e4-a370-535487b2a4cc None None] MFA Rules not processed for user `8bfca4d302aa43249609513f35d55d7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:30:56.986596 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-2682951d-ec71-40e4-a370-535487b2a4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:30:57.019081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:30:57.050021 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 694/1401] 10.222.0.15 () {58 vars in 1059 bytes} [Wed Jun 15 19:30:56 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:30:57.060811 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb5408-529c-4c6c-919e-810aaed3c626 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:30:57.066556 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=55 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:30:57.073616 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb5408-529c-4c6c-919e-810aaed3c626 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:30:57.079272 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78978c5c-06ba-435a-a1fc-35492f423860 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfca4d302aa43249609513f35d55d7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:30:57.079913 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78978c5c-06ba-435a-a1fc-35492f423860 service glance] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:57.080313 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78978c5c-06ba-435a-a1fc-35492f423860 service glance] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:57.080593 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78978c5c-06ba-435a-a1fc-35492f423860 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:57.091735 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78978c5c-06ba-435a-a1fc-35492f423860 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:30:57.093094 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78978c5c-06ba-435a-a1fc-35492f423860 service glance] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:30:57.102581 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 708/1402] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:30:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:30:57.865395 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54faa0eb-2d2b-471a-bdf9-3a5dcf454687 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:57.865892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54faa0eb-2d2b-471a-bdf9-3a5dcf454687 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:57.866008 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54faa0eb-2d2b-471a-bdf9-3a5dcf454687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:57.917884 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-54faa0eb-2d2b-471a-bdf9-3a5dcf454687 None None] MFA Rules not processed for user `67aa048d409c46e9a729aec9d1b2286e`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:30:57.919150 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-54faa0eb-2d2b-471a-bdf9-3a5dcf454687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:30:57.983790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 695/1403] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Jun 15 19:30:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:30:58.024973 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0694f32-512f-41a6-82cc-ab774c5f370b None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:58.026013 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0694f32-512f-41a6-82cc-ab774c5f370b None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:58.026247 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0694f32-512f-41a6-82cc-ab774c5f370b None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:58.027738 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 709/1404] 10.222.0.15 () {54 vars in 942 bytes} [Wed Jun 15 19:30:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:30:58.034172 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123f829-a9d4-40c6-a200-3957644a61ef None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:58.034540 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123f829-a9d4-40c6-a200-3957644a61ef None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:58.034896 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123f829-a9d4-40c6-a200-3957644a61ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:58.083288 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-d123f829-a9d4-40c6-a200-3957644a61ef None None] MFA Rules not processed for user `176d5775a70f4524a9b7fe1ce9d426ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:30:58.084448 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d123f829-a9d4-40c6-a200-3957644a61ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:30:58.163111 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 696/1405] 10.222.0.15 () {58 vars in 1039 bytes} [Wed Jun 15 19:30:58 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:30:58.173104 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0694f32-512f-41a6-82cc-ab774c5f370b None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:30:58.187058 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0694f32-512f-41a6-82cc-ab774c5f370b None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:30:58.191958 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e636544c-8517-4087-b10f-b194a9859dbf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176d5775a70f4524a9b7fe1ce9d426ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:30:58.192598 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e636544c-8517-4087-b10f-b194a9859dbf service swift] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:58.192890 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e636544c-8517-4087-b10f-b194a9859dbf service swift] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:58.193170 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e636544c-8517-4087-b10f-b194a9859dbf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:58.201125 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e636544c-8517-4087-b10f-b194a9859dbf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:30:58.202742 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e636544c-8517-4087-b10f-b194a9859dbf service swift] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:30:58.210796 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 710/1406] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Jun 15 19:30:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 19:30:58.714611 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaaac25-71c9-4c70-8f47-5ca9f5e30c30 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:58.714808 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaaac25-71c9-4c70-8f47-5ca9f5e30c30 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:58.715076 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaaac25-71c9-4c70-8f47-5ca9f5e30c30 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:58.716458 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 697/1407] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:30:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:30:58.721483 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb5ab5-a246-47b5-9761-17b3f2012f0d None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:58.721777 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb5ab5-a246-47b5-9761-17b3f2012f0d None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:58.722060 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb5ab5-a246-47b5-9761-17b3f2012f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:58.743836 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-c8eb5ab5-a246-47b5-9761-17b3f2012f0d None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:30:58.745059 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-c8eb5ab5-a246-47b5-9761-17b3f2012f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:30:58.753444 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 711/1408] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:30:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:30:58.797885 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaaac25-71c9-4c70-8f47-5ca9f5e30c30 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:30:58.812053 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaaac25-71c9-4c70-8f47-5ca9f5e30c30 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:30:58.817461 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1324c33-4aa5-48d5-9966-de5ac33667ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:30:58.818118 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1324c33-4aa5-48d5-9966-de5ac33667ca service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:30:58.819698 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1324c33-4aa5-48d5-9966-de5ac33667ca service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:30:58.819698 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1324c33-4aa5-48d5-9966-de5ac33667ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:30:58.828880 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1324c33-4aa5-48d5-9966-de5ac33667ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:30:58.830499 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1324c33-4aa5-48d5-9966-de5ac33667ca service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:30:58.835542 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:30:58.853582 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 698/1409] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:30:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:31:08.738789 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=14 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:31:20.484934 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdde59d-e1dc-4000-8fc2-834fcac1fd05 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:20.485586 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdde59d-e1dc-4000-8fc2-834fcac1fd05 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:20.485586 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdde59d-e1dc-4000-8fc2-834fcac1fd05 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:20.488019 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 712/1410] 10.222.0.15 () {54 vars in 918 bytes} [Wed Jun 15 19:31:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:31:20.492711 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e9354-1ddc-49d9-9595-e00733dc5114 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:20.493130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e9354-1ddc-49d9-9595-e00733dc5114 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:20.493275 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e9354-1ddc-49d9-9595-e00733dc5114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:20.512441 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-aa1e9354-1ddc-49d9-9595-e00733dc5114 None None] MFA Rules not processed for user `cb0ec2ef6c4748e181f1963e7dfe30b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:31:20.513644 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-aa1e9354-1ddc-49d9-9595-e00733dc5114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:31:20.522174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 699/1411] 10.222.0.15 () {58 vars in 1015 bytes} [Wed Jun 15 19:31:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:31:20.536140 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec980c0-ae2f-4c81-b098-1a50f0beafea None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:20.536754 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec980c0-ae2f-4c81-b098-1a50f0beafea None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:20.537036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec980c0-ae2f-4c81-b098-1a50f0beafea None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:20.538435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 713/1412] 10.222.0.15 () {54 vars in 918 bytes} [Wed Jun 15 19:31:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:31:20.542943 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea86096-b189-4747-9a59-25bac9c1e219 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:20.543237 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea86096-b189-4747-9a59-25bac9c1e219 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:20.543516 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea86096-b189-4747-9a59-25bac9c1e219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:20.561330 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-0ea86096-b189-4747-9a59-25bac9c1e219 None None] MFA Rules not processed for user `3e74ac44e5814122a3226ac023f3688a`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:31:20.562548 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0ea86096-b189-4747-9a59-25bac9c1e219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:31:20.571516 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 700/1413] 10.222.0.15 () {58 vars in 1015 bytes} [Wed Jun 15 19:31:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:31:23.144719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e184b458-4fe8-4aa6-9986-da6f6fcba504 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:23.145262 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e184b458-4fe8-4aa6-9986-da6f6fcba504 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:23.145262 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e184b458-4fe8-4aa6-9986-da6f6fcba504 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:23.146610 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 714/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 19:31:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:31:23.152343 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c354e-621d-436b-9158-6f657ac6da52 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:23.152783 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c354e-621d-436b-9158-6f657ac6da52 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:23.152902 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c354e-621d-436b-9158-6f657ac6da52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:23.170222 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-082c354e-621d-436b-9158-6f657ac6da52 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:31:23.171414 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-082c354e-621d-436b-9158-6f657ac6da52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:31:23.179135 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 701/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 19:31:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:31:23.222258 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e184b458-4fe8-4aa6-9986-da6f6fcba504 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:31:23.236505 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e184b458-4fe8-4aa6-9986-da6f6fcba504 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:31:23.241709 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35595e9d-525b-4ca8-9143-d14cf39c2ec9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:31:23.242485 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35595e9d-525b-4ca8-9143-d14cf39c2ec9 service nova] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:23.242766 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35595e9d-525b-4ca8-9143-d14cf39c2ec9 service nova] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:23.243037 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35595e9d-525b-4ca8-9143-d14cf39c2ec9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:23.252273 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35595e9d-525b-4ca8-9143-d14cf39c2ec9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:31:23.253879 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35595e9d-525b-4ca8-9143-d14cf39c2ec9 service nova] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:31:23.264011 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 715/1416] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:31:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:31:24.222832 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d0fc7-a3f3-4303-88cb-960ad5ff3475 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:24.223377 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d0fc7-a3f3-4303-88cb-960ad5ff3475 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:24.223377 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d0fc7-a3f3-4303-88cb-960ad5ff3475 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:24.224794 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 702/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 19:31:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:31:24.229736 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc05a58-955c-4eb8-b01f-4c24bbea85f4 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:24.230191 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc05a58-955c-4eb8-b01f-4c24bbea85f4 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:24.230332 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc05a58-955c-4eb8-b01f-4c24bbea85f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:24.248277 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-cdc05a58-955c-4eb8-b01f-4c24bbea85f4 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:31:24.249464 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-cdc05a58-955c-4eb8-b01f-4c24bbea85f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:31:24.257398 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 716/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 19:31:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:31:24.414810 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d0fc7-a3f3-4303-88cb-960ad5ff3475 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:31:24.429479 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d0fc7-a3f3-4303-88cb-960ad5ff3475 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:31:24.435472 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7339af8-00c9-46c4-abd1-7887c05d5997 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:31:24.436103 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7339af8-00c9-46c4-abd1-7887c05d5997 service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:24.436390 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7339af8-00c9-46c4-abd1-7887c05d5997 service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:24.436668 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7339af8-00c9-46c4-abd1-7887c05d5997 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:24.444686 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7339af8-00c9-46c4-abd1-7887c05d5997 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:31:24.446289 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7339af8-00c9-46c4-abd1-7887c05d5997 service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:31:24.456128 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 703/1419] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:31:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:31:25.373110 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a526e4b-74f7-4ba4-a4fa-5edb7d9feb23 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:25.373548 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a526e4b-74f7-4ba4-a4fa-5edb7d9feb23 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:25.373646 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a526e4b-74f7-4ba4-a4fa-5edb7d9feb23 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:25.375006 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 717/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 19:31:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:31:25.380137 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950465d-9bc4-478c-bced-bc30e60ac1d2 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:25.380528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950465d-9bc4-478c-bced-bc30e60ac1d2 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:25.380703 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950465d-9bc4-478c-bced-bc30e60ac1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:25.398934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-8950465d-9bc4-478c-bced-bc30e60ac1d2 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:31:25.400132 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8950465d-9bc4-478c-bced-bc30e60ac1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:31:25.408547 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 704/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 19:31:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:31:25.560358 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a526e4b-74f7-4ba4-a4fa-5edb7d9feb23 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:31:25.574928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a526e4b-74f7-4ba4-a4fa-5edb7d9feb23 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:31:25.580244 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92012ae-72e6-4687-9ef3-08089948c7b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:31:25.580877 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92012ae-72e6-4687-9ef3-08089948c7b7 service nova] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:25.581163 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92012ae-72e6-4687-9ef3-08089948c7b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:25.581442 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92012ae-72e6-4687-9ef3-08089948c7b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:25.589430 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92012ae-72e6-4687-9ef3-08089948c7b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:31:25.591037 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92012ae-72e6-4687-9ef3-08089948c7b7 service nova] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:31:25.600555 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 718/1422] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:31:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:31:26.498688 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b06409-73d6-4ac7-8e7a-5c10793b4d6f None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:26.499245 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b06409-73d6-4ac7-8e7a-5c10793b4d6f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:26.499245 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b06409-73d6-4ac7-8e7a-5c10793b4d6f None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:26.500623 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 705/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 19:31:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:31:26.505665 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d08853-8631-49ed-857e-d52ab49b3566 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:26.506080 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d08853-8631-49ed-857e-d52ab49b3566 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:26.506250 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d08853-8631-49ed-857e-d52ab49b3566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:26.524408 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-45d08853-8631-49ed-857e-d52ab49b3566 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:31:26.525600 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-45d08853-8631-49ed-857e-d52ab49b3566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:31:26.533386 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 719/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 19:31:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:31:26.684403 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b06409-73d6-4ac7-8e7a-5c10793b4d6f None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:31:26.698475 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b06409-73d6-4ac7-8e7a-5c10793b4d6f None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:31:26.703818 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36862f68-9c88-4e6d-99ee-a466b43a5d55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:31:26.704449 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36862f68-9c88-4e6d-99ee-a466b43a5d55 service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:26.704730 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36862f68-9c88-4e6d-99ee-a466b43a5d55 service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:26.705035 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36862f68-9c88-4e6d-99ee-a466b43a5d55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:26.713090 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36862f68-9c88-4e6d-99ee-a466b43a5d55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:31:26.714706 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36862f68-9c88-4e6d-99ee-a466b43a5d55 service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:31:26.723977 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 706/1425] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:31:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:31:27.632118 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8aa90-7b3b-4cb2-92ad-11e5f59565f8 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:27.632686 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8aa90-7b3b-4cb2-92ad-11e5f59565f8 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:27.632686 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8aa90-7b3b-4cb2-92ad-11e5f59565f8 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:27.634049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 720/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 19:31:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:31:27.638934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e1a72-eee3-4eee-b943-debcc4e00325 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:27.639558 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e1a72-eee3-4eee-b943-debcc4e00325 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:27.639558 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e1a72-eee3-4eee-b943-debcc4e00325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:27.657042 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-da7e1a72-eee3-4eee-b943-debcc4e00325 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:31:27.658255 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-da7e1a72-eee3-4eee-b943-debcc4e00325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:31:27.666400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 707/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 19:31:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:31:27.815218 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc8aa90-7b3b-4cb2-92ad-11e5f59565f8 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:31:27.830457 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc8aa90-7b3b-4cb2-92ad-11e5f59565f8 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:31:27.835936 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c86af-e753-4957-8570-e5f2851ea5e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:31:27.836598 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c86af-e753-4957-8570-e5f2851ea5e7 service nova] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:27.836881 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c86af-e753-4957-8570-e5f2851ea5e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:27.837157 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c86af-e753-4957-8570-e5f2851ea5e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:27.845752 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818c86af-e753-4957-8570-e5f2851ea5e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:31:27.847421 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818c86af-e753-4957-8570-e5f2851ea5e7 service nova] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:31:27.857537 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 721/1428] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:31:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:31:28.700169 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ce4e0-ae04-4c41-a9d6-2fd52b6ad0ef None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:28.700730 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ce4e0-ae04-4c41-a9d6-2fd52b6ad0ef None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:28.700730 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ce4e0-ae04-4c41-a9d6-2fd52b6ad0ef None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:28.702005 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 708/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 19:31:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:31:28.706855 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21bdb7-a9f1-405e-b320-683863d9c6ce None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:28.707256 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21bdb7-a9f1-405e-b320-683863d9c6ce None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:28.707416 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21bdb7-a9f1-405e-b320-683863d9c6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:28.725204 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-fa21bdb7-a9f1-405e-b320-683863d9c6ce None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:31:28.726412 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-fa21bdb7-a9f1-405e-b320-683863d9c6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:31:28.734619 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 722/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 19:31:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:31:28.883894 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ce4e0-ae04-4c41-a9d6-2fd52b6ad0ef None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:31:28.898432 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ce4e0-ae04-4c41-a9d6-2fd52b6ad0ef None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:31:28.904360 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e01c6-3a8f-4f12-839e-7071c7a6566b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:31:28.904989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e01c6-3a8f-4f12-839e-7071c7a6566b service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:28.905268 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e01c6-3a8f-4f12-839e-7071c7a6566b service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:28.905541 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e01c6-3a8f-4f12-839e-7071c7a6566b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:28.913450 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e01c6-3a8f-4f12-839e-7071c7a6566b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:31:28.915050 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e01c6-3a8f-4f12-839e-7071c7a6566b service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:31:28.924195 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 709/1431] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:31:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:31:29.757728 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ebbcb-076e-4df8-a7e9-4b54a3cbf1ef None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:29.758303 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ebbcb-076e-4df8-a7e9-4b54a3cbf1ef None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:29.758303 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ebbcb-076e-4df8-a7e9-4b54a3cbf1ef None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:29.760218 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 723/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 19:31:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:31:29.764930 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9697e8-760c-416b-8a28-5fbad041f072 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:29.765377 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9697e8-760c-416b-8a28-5fbad041f072 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:29.765478 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9697e8-760c-416b-8a28-5fbad041f072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:29.783755 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-2c9697e8-760c-416b-8a28-5fbad041f072 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:31:29.784951 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-2c9697e8-760c-416b-8a28-5fbad041f072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:31:29.792693 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 710/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 19:31:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:31:29.941538 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310ebbcb-076e-4df8-a7e9-4b54a3cbf1ef None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:31:29.956152 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310ebbcb-076e-4df8-a7e9-4b54a3cbf1ef None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:31:29.961512 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33e1644-ed12-4794-82b1-c81f68e36a60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:31:29.962168 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e1644-ed12-4794-82b1-c81f68e36a60 service nova] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:29.962455 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e1644-ed12-4794-82b1-c81f68e36a60 service nova] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:29.963287 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e1644-ed12-4794-82b1-c81f68e36a60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:29.971255 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33e1644-ed12-4794-82b1-c81f68e36a60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:31:29.972826 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33e1644-ed12-4794-82b1-c81f68e36a60 service nova] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:31:29.982033 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 724/1434] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:31:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:31:30.820402 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35ae85-2726-4694-b8a7-a76ae9ec6e80 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:30.820960 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35ae85-2726-4694-b8a7-a76ae9ec6e80 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:30.820960 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35ae85-2726-4694-b8a7-a76ae9ec6e80 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:30.822935 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 711/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 19:31:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:31:30.827681 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4c94d-c1ee-496d-a28e-42d7e8b624f4 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:30.828095 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4c94d-c1ee-496d-a28e-42d7e8b624f4 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:30.828247 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4c94d-c1ee-496d-a28e-42d7e8b624f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:30.845904 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-a5a4c94d-c1ee-496d-a28e-42d7e8b624f4 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:31:30.847138 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-a5a4c94d-c1ee-496d-a28e-42d7e8b624f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:31:30.854900 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 725/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 19:31:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:31:31.003558 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a35ae85-2726-4694-b8a7-a76ae9ec6e80 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:31:31.018320 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a35ae85-2726-4694-b8a7-a76ae9ec6e80 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:31:31.023693 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc34bee7-b2b1-4cf1-af59-8cc1feb3de91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:31:31.024320 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34bee7-b2b1-4cf1-af59-8cc1feb3de91 service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:31.024622 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34bee7-b2b1-4cf1-af59-8cc1feb3de91 service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:31.024884 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34bee7-b2b1-4cf1-af59-8cc1feb3de91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:31.032975 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc34bee7-b2b1-4cf1-af59-8cc1feb3de91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:31:31.034567 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc34bee7-b2b1-4cf1-af59-8cc1feb3de91 service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:31:31.043816 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 712/1437] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:31:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:31:31.882949 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875774f6-8097-4988-9821-86061ba945dd None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:31.883493 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875774f6-8097-4988-9821-86061ba945dd None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:31.883493 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875774f6-8097-4988-9821-86061ba945dd None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:31.884941 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 726/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 19:31:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:31:31.889875 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26098156-4187-42d8-86f0-57394a1b2c99 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:31.890342 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26098156-4187-42d8-86f0-57394a1b2c99 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:31.890466 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26098156-4187-42d8-86f0-57394a1b2c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:31.908046 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-26098156-4187-42d8-86f0-57394a1b2c99 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:31:31.909227 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-26098156-4187-42d8-86f0-57394a1b2c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:31:31.917481 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 713/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 19:31:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:31:32.066822 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875774f6-8097-4988-9821-86061ba945dd None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:31:32.081522 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875774f6-8097-4988-9821-86061ba945dd None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:31:32.086872 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebba89d-4671-4021-af93-605e4414db31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:31:32.087499 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebba89d-4671-4021-af93-605e4414db31 service nova] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:32.088349 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebba89d-4671-4021-af93-605e4414db31 service nova] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:32.088623 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebba89d-4671-4021-af93-605e4414db31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:32.096506 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebba89d-4671-4021-af93-605e4414db31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:31:32.098070 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebba89d-4671-4021-af93-605e4414db31 service nova] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:31:32.107231 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 727/1440] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:31:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:31:34.095892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dcbada-09a7-4337-bcff-58ac51dcea79 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:34.096454 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dcbada-09a7-4337-bcff-58ac51dcea79 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:34.096454 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dcbada-09a7-4337-bcff-58ac51dcea79 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:34.097806 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 714/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 19:31:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:31:34.102850 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa925d98-9552-4a0f-acff-e5b7d8763d3c None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:34.103300 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa925d98-9552-4a0f-acff-e5b7d8763d3c None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:34.103976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa925d98-9552-4a0f-acff-e5b7d8763d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:34.121312 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-fa925d98-9552-4a0f-acff-e5b7d8763d3c None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:31:34.122570 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-fa925d98-9552-4a0f-acff-e5b7d8763d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:31:34.130442 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 728/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 19:31:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:31:34.167972 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dcbada-09a7-4337-bcff-58ac51dcea79 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:31:34.178659 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dcbada-09a7-4337-bcff-58ac51dcea79 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:31:34.182786 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292389b2-4217-43fa-91ea-29bf2c26117d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:31:34.183411 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292389b2-4217-43fa-91ea-29bf2c26117d service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:34.183695 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292389b2-4217-43fa-91ea-29bf2c26117d service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:34.183966 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292389b2-4217-43fa-91ea-29bf2c26117d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:34.193098 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292389b2-4217-43fa-91ea-29bf2c26117d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:31:34.194565 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292389b2-4217-43fa-91ea-29bf2c26117d service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:31:34.206520 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 715/1443] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:31:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:31:36.518538 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335de53f-1b28-4484-8f6a-e688865a324e None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:36.519148 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335de53f-1b28-4484-8f6a-e688865a324e None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:36.519148 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335de53f-1b28-4484-8f6a-e688865a324e None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:36.520465 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 729/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 19:31:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:31:36.526286 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e4773-5682-4333-bcd4-61c06f38a183 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:36.526721 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e4773-5682-4333-bcd4-61c06f38a183 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:36.526825 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e4773-5682-4333-bcd4-61c06f38a183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:36.545040 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-d31e4773-5682-4333-bcd4-61c06f38a183 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:31:36.546621 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d31e4773-5682-4333-bcd4-61c06f38a183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:31:36.557765 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 716/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 19:31:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:31:36.596625 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335de53f-1b28-4484-8f6a-e688865a324e None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:31:36.598282 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-335de53f-1b28-4484-8f6a-e688865a324e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:31:36.610884 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335de53f-1b28-4484-8f6a-e688865a324e None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:31:36.614394 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb3ff7-5359-459a-ad4f-336f40b947db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:31:36.615020 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb3ff7-5359-459a-ad4f-336f40b947db service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:36.615301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb3ff7-5359-459a-ad4f-336f40b947db service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:36.615576 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb3ff7-5359-459a-ad4f-336f40b947db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:36.625542 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebb3ff7-5359-459a-ad4f-336f40b947db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:31:36.627282 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebb3ff7-5359-459a-ad4f-336f40b947db service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:31:36.636922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 730/1446] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:31:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:31:39.049514 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876a916-c4e6-428d-b4ca-636d19da498d None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:39.050051 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876a916-c4e6-428d-b4ca-636d19da498d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:39.050051 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876a916-c4e6-428d-b4ca-636d19da498d None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:39.051454 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 717/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 19:31:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:31:39.057294 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cfe31a-2a17-48c7-acf5-a2c9a941dbfc None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:39.057721 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cfe31a-2a17-48c7-acf5-a2c9a941dbfc None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:39.057852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cfe31a-2a17-48c7-acf5-a2c9a941dbfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:39.075592 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-83cfe31a-2a17-48c7-acf5-a2c9a941dbfc None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:31:39.076767 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-83cfe31a-2a17-48c7-acf5-a2c9a941dbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:31:39.084507 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 731/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 19:31:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:31:39.121797 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876a916-c4e6-428d-b4ca-636d19da498d None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:31:39.132846 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876a916-c4e6-428d-b4ca-636d19da498d None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:31:39.136834 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb5d9bf-061e-41ce-8f7f-2fee695ae6cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:31:39.137462 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5d9bf-061e-41ce-8f7f-2fee695ae6cc service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:39.137744 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5d9bf-061e-41ce-8f7f-2fee695ae6cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:39.138015 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5d9bf-061e-41ce-8f7f-2fee695ae6cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:39.146729 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb5d9bf-061e-41ce-8f7f-2fee695ae6cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:31:39.148152 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb5d9bf-061e-41ce-8f7f-2fee695ae6cc service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:31:39.157395 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 718/1449] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:31:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:31:41.381020 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600fd698-fb5c-47f4-923c-dc8d8c30d3ac None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:41.381517 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600fd698-fb5c-47f4-923c-dc8d8c30d3ac None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:41.381517 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600fd698-fb5c-47f4-923c-dc8d8c30d3ac None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:41.382956 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 732/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 19:31:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:31:41.389829 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add34ac2-4a74-4da7-a236-5f94ee818bae None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:41.390298 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add34ac2-4a74-4da7-a236-5f94ee818bae None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:41.390728 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add34ac2-4a74-4da7-a236-5f94ee818bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:41.408135 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-add34ac2-4a74-4da7-a236-5f94ee818bae None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:31:41.409310 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-add34ac2-4a74-4da7-a236-5f94ee818bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:31:41.416923 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 719/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 19:31:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:31:41.455394 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600fd698-fb5c-47f4-923c-dc8d8c30d3ac None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:31:41.466027 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600fd698-fb5c-47f4-923c-dc8d8c30d3ac None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:31:41.469490 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2509e5-7182-4999-979e-4a62a88dd5b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:31:41.470152 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2509e5-7182-4999-979e-4a62a88dd5b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:41.470438 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2509e5-7182-4999-979e-4a62a88dd5b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:41.470715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2509e5-7182-4999-979e-4a62a88dd5b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:41.479597 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2509e5-7182-4999-979e-4a62a88dd5b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:31:41.481049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2509e5-7182-4999-979e-4a62a88dd5b3 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:31:41.490476 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 733/1452] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:31:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:31:43.631143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b1586-5a3d-48a5-9824-f11b003c3096 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:43.631678 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b1586-5a3d-48a5-9824-f11b003c3096 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:43.631678 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b1586-5a3d-48a5-9824-f11b003c3096 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:43.633042 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 720/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 19:31:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:31:43.638178 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962540f-7ae8-4ea7-92fa-9177e061bba2 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:43.638583 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962540f-7ae8-4ea7-92fa-9177e061bba2 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:43.638740 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962540f-7ae8-4ea7-92fa-9177e061bba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:43.656613 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-d962540f-7ae8-4ea7-92fa-9177e061bba2 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:31:43.657808 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-d962540f-7ae8-4ea7-92fa-9177e061bba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:31:43.665507 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 734/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 19:31:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:31:43.702682 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b1586-5a3d-48a5-9824-f11b003c3096 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:31:43.713710 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b1586-5a3d-48a5-9824-f11b003c3096 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:31:43.717207 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdbe558-7e18-433d-a9d6-6c6b2bfd3a18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:31:43.717835 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdbe558-7e18-433d-a9d6-6c6b2bfd3a18 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:43.718218 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdbe558-7e18-433d-a9d6-6c6b2bfd3a18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:43.718500 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdbe558-7e18-433d-a9d6-6c6b2bfd3a18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:43.727205 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdbe558-7e18-433d-a9d6-6c6b2bfd3a18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:31:43.729178 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdbe558-7e18-433d-a9d6-6c6b2bfd3a18 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:31:43.738494 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 721/1455] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:31:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:31:45.895784 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a849a6-04b8-4e54-a26f-54e3e956016c None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:45.896368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a849a6-04b8-4e54-a26f-54e3e956016c None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:45.896368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a849a6-04b8-4e54-a26f-54e3e956016c None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:45.897723 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 735/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 19:31:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:31:45.903537 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88763920-84e5-41c2-902f-c8362f397e63 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:45.903981 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88763920-84e5-41c2-902f-c8362f397e63 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:45.904091 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88763920-84e5-41c2-902f-c8362f397e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:45.922711 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-88763920-84e5-41c2-902f-c8362f397e63 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:31:45.923904 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-88763920-84e5-41c2-902f-c8362f397e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:31:45.932188 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 722/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 19:31:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:31:45.969461 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a849a6-04b8-4e54-a26f-54e3e956016c None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:31:45.980424 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a849a6-04b8-4e54-a26f-54e3e956016c None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:31:45.983816 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014302f-f3b2-41a3-b6d4-e5ab956ab2bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:31:45.984438 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014302f-f3b2-41a3-b6d4-e5ab956ab2bc service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:45.984717 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014302f-f3b2-41a3-b6d4-e5ab956ab2bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:45.984990 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014302f-f3b2-41a3-b6d4-e5ab956ab2bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:45.993635 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e014302f-f3b2-41a3-b6d4-e5ab956ab2bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:31:45.995070 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e014302f-f3b2-41a3-b6d4-e5ab956ab2bc service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:31:46.004473 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 736/1458] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:31:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:31:48.187944 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09630809-64f0-4e68-9178-7e267d79a602 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:48.188511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09630809-64f0-4e68-9178-7e267d79a602 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:48.188511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09630809-64f0-4e68-9178-7e267d79a602 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:48.189861 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 723/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 19:31:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:31:48.195122 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf809407-5479-440d-92d1-7795b594573d None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:48.195552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf809407-5479-440d-92d1-7795b594573d None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:48.195715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf809407-5479-440d-92d1-7795b594573d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:48.213616 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-cf809407-5479-440d-92d1-7795b594573d None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:31:48.214840 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-cf809407-5479-440d-92d1-7795b594573d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:31:48.222562 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 737/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 19:31:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:31:48.259192 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09630809-64f0-4e68-9178-7e267d79a602 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:31:48.270671 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09630809-64f0-4e68-9178-7e267d79a602 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:31:48.274143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687cfbd4-c8f6-4f29-b290-04606f23af5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:31:48.274778 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687cfbd4-c8f6-4f29-b290-04606f23af5f service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:48.275061 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687cfbd4-c8f6-4f29-b290-04606f23af5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:48.275337 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687cfbd4-c8f6-4f29-b290-04606f23af5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:48.287392 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687cfbd4-c8f6-4f29-b290-04606f23af5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:31:48.288836 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687cfbd4-c8f6-4f29-b290-04606f23af5f service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:31:48.298576 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 724/1461] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:31:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:31:50.444957 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f931957-99c2-41d4-a962-3bf63a338f21 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:50.445516 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f931957-99c2-41d4-a962-3bf63a338f21 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:50.445516 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f931957-99c2-41d4-a962-3bf63a338f21 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:50.447556 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 738/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 19:31:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:31:50.452724 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faffabf-0eae-4a17-ac5c-456795d538a9 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:50.453162 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faffabf-0eae-4a17-ac5c-456795d538a9 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:50.453261 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faffabf-0eae-4a17-ac5c-456795d538a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:50.459357 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=79 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:31:50.471643 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-9faffabf-0eae-4a17-ac5c-456795d538a9 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:31:50.472821 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-9faffabf-0eae-4a17-ac5c-456795d538a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:31:50.480574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 725/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 19:31:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:31:50.517622 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f931957-99c2-41d4-a962-3bf63a338f21 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:31:50.523001 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=73 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:31:50.530732 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f931957-99c2-41d4-a962-3bf63a338f21 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:31:50.534269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b7aad7-e80a-4c53-9acb-11c22dc107d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:31:50.534907 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b7aad7-e80a-4c53-9acb-11c22dc107d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:50.535189 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b7aad7-e80a-4c53-9acb-11c22dc107d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:50.535464 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b7aad7-e80a-4c53-9acb-11c22dc107d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:50.544525 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b7aad7-e80a-4c53-9acb-11c22dc107d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:31:50.547623 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b7aad7-e80a-4c53-9acb-11c22dc107d3 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:31:50.560621 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 739/1464] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:31:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:31:52.654788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a5110-8569-41cf-8c83-533a166e28b1 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:52.655312 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a5110-8569-41cf-8c83-533a166e28b1 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:52.655312 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a5110-8569-41cf-8c83-533a166e28b1 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:52.656609 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 726/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 19:31:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:31:52.661673 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d880f09-cce4-4916-8609-77305c4f87ea None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:52.662055 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d880f09-cce4-4916-8609-77305c4f87ea None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:52.662276 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d880f09-cce4-4916-8609-77305c4f87ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:52.680034 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-2d880f09-cce4-4916-8609-77305c4f87ea None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:31:52.681260 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-2d880f09-cce4-4916-8609-77305c4f87ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:31:52.688895 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 740/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 19:31:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:31:52.724062 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a5110-8569-41cf-8c83-533a166e28b1 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:31:52.735192 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a5110-8569-41cf-8c83-533a166e28b1 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:31:52.738635 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db896e49-dd79-4a76-b6f1-25c541990e40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:31:52.739264 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db896e49-dd79-4a76-b6f1-25c541990e40 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:52.739543 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db896e49-dd79-4a76-b6f1-25c541990e40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:52.739817 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db896e49-dd79-4a76-b6f1-25c541990e40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:52.748453 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db896e49-dd79-4a76-b6f1-25c541990e40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:31:52.749930 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db896e49-dd79-4a76-b6f1-25c541990e40 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:31:52.759454 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 727/1467] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:31:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:31:54.872286 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ed8ec-8ee6-49ef-9523-48314c6974ee None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:54.872854 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ed8ec-8ee6-49ef-9523-48314c6974ee None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:54.872854 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ed8ec-8ee6-49ef-9523-48314c6974ee None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:54.874336 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 741/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 19:31:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:31:54.879953 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99802bd-f1f8-45a9-aa24-821bfc01ad4f None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:54.880350 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99802bd-f1f8-45a9-aa24-821bfc01ad4f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:54.880502 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99802bd-f1f8-45a9-aa24-821bfc01ad4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:54.898419 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c99802bd-f1f8-45a9-aa24-821bfc01ad4f None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:31:54.899614 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c99802bd-f1f8-45a9-aa24-821bfc01ad4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:31:54.907204 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 728/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 19:31:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:31:54.942648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3ed8ec-8ee6-49ef-9523-48314c6974ee None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:31:54.953864 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3ed8ec-8ee6-49ef-9523-48314c6974ee None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:31:54.957443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23656a-503e-4aaf-a91b-43319556ffa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:31:54.958077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23656a-503e-4aaf-a91b-43319556ffa6 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:31:54.958386 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23656a-503e-4aaf-a91b-43319556ffa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:31:54.958666 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23656a-503e-4aaf-a91b-43319556ffa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:31:54.967433 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c23656a-503e-4aaf-a91b-43319556ffa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:31:54.968857 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c23656a-503e-4aaf-a91b-43319556ffa6 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:31:54.978175 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 742/1470] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:31:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:32:04.893399 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=11 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:32:04.974260 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=14 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:32:14.814232 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57567a9b-6bfc-4228-b4a7-4dfecdbffa2f None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:32:14.814820 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57567a9b-6bfc-4228-b4a7-4dfecdbffa2f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:32:14.814820 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57567a9b-6bfc-4228-b4a7-4dfecdbffa2f None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:32:14.816178 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 729/1471] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:32:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:32:14.821229 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5988fd0-e1ad-4849-84ab-9b7db1158e8c None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:32:14.822213 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5988fd0-e1ad-4849-84ab-9b7db1158e8c None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:32:14.822367 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5988fd0-e1ad-4849-84ab-9b7db1158e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:32:14.840013 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-d5988fd0-e1ad-4849-84ab-9b7db1158e8c None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:32:14.841201 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-d5988fd0-e1ad-4849-84ab-9b7db1158e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:32:14.848866 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 743/1472] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:32:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:32:14.894514 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e59ef9-3584-4ef4-98fd-37ca291a97ce None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:32:14.894798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e59ef9-3584-4ef4-98fd-37ca291a97ce None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:32:14.895071 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e59ef9-3584-4ef4-98fd-37ca291a97ce None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:32:14.896517 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 730/1473] 10.222.0.15 () {54 vars in 962 bytes} [Wed Jun 15 19:32:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:32:14.901919 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33179c48-32e9-42e2-a2cf-a5368b82586a None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:32:14.902248 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33179c48-32e9-42e2-a2cf-a5368b82586a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:32:14.902533 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33179c48-32e9-42e2-a2cf-a5368b82586a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:32:14.920612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-33179c48-32e9-42e2-a2cf-a5368b82586a None None] MFA Rules not processed for user `8bfca4d302aa43249609513f35d55d7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:32:14.921799 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-33179c48-32e9-42e2-a2cf-a5368b82586a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:32:14.928714 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 744/1474] 10.222.0.15 () {58 vars in 1059 bytes} [Wed Jun 15 19:32:14 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:32:14.936405 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e59ef9-3584-4ef4-98fd-37ca291a97ce None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:32:14.947381 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e59ef9-3584-4ef4-98fd-37ca291a97ce None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:32:14.950885 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c689e1-e0fd-4d85-af40-903f267a7257 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfca4d302aa43249609513f35d55d7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:32:14.951506 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c689e1-e0fd-4d85-af40-903f267a7257 service glance] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:32:14.951784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c689e1-e0fd-4d85-af40-903f267a7257 service glance] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:32:14.952054 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c689e1-e0fd-4d85-af40-903f267a7257 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:32:14.960692 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c689e1-e0fd-4d85-af40-903f267a7257 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:32:14.962697 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c689e1-e0fd-4d85-af40-903f267a7257 service glance] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:32:14.971783 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 731/1475] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:32:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:32:17.151350 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae6b77-c9de-4f41-b0a6-8551ccaffafa None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:32:17.151830 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae6b77-c9de-4f41-b0a6-8551ccaffafa None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:32:17.151929 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae6b77-c9de-4f41-b0a6-8551ccaffafa None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:32:17.153263 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 745/1476] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:32:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:32:17.158380 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4226d6d-853a-4228-ad8f-555ec789f7b7 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:32:17.158770 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4226d6d-853a-4228-ad8f-555ec789f7b7 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:32:17.158935 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4226d6d-853a-4228-ad8f-555ec789f7b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:32:17.176155 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-d4226d6d-853a-4228-ad8f-555ec789f7b7 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:32:17.177325 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d4226d6d-853a-4228-ad8f-555ec789f7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:32:17.184997 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 732/1477] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:32:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:32:17.207785 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae6b77-c9de-4f41-b0a6-8551ccaffafa None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:32:17.220415 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae6b77-c9de-4f41-b0a6-8551ccaffafa None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:32:17.224842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ead8c3-02f3-4de4-98ed-9216fd384d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:32:17.225475 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ead8c3-02f3-4de4-98ed-9216fd384d8e admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:32:17.225754 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ead8c3-02f3-4de4-98ed-9216fd384d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:32:17.226025 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ead8c3-02f3-4de4-98ed-9216fd384d8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:32:17.226777 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ead8c3-02f3-4de4-98ed-9216fd384d8e admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:32:17.228537 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: WARNING py.warnings [None req-c0ead8c3-02f3-4de4-98ed-9216fd384d8e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 19:32:17.228537 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: warnings.warn(msg) Jun 15 19:32:17.228537 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: Jun 15 19:32:17.229664 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ead8c3-02f3-4de4-98ed-9216fd384d8e admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:32:17.255151 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 746/1478] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Jun 15 19:32:17 2022] GET /identity/v3/projects => generated 2553 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:32:19.390711 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26986dd1-0ffe-4ede-973e-cc2991e2d0b6 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:32:19.391141 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26986dd1-0ffe-4ede-973e-cc2991e2d0b6 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:32:19.391372 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26986dd1-0ffe-4ede-973e-cc2991e2d0b6 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:32:19.392840 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 733/1479] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:32:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:32:19.397973 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1cd05f-856a-49c0-93a7-ac4e26cacf0d None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:32:19.398502 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1cd05f-856a-49c0-93a7-ac4e26cacf0d None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:32:19.398622 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1cd05f-856a-49c0-93a7-ac4e26cacf0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:32:19.417473 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-ac1cd05f-856a-49c0-93a7-ac4e26cacf0d None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:32:19.418709 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-ac1cd05f-856a-49c0-93a7-ac4e26cacf0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:32:19.426772 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 747/1480] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:32:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:32:19.470300 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26986dd1-0ffe-4ede-973e-cc2991e2d0b6 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:32:19.485941 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26986dd1-0ffe-4ede-973e-cc2991e2d0b6 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:32:19.491630 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511330a3-15e3-43af-9baf-9c12cac734ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:32:19.492296 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511330a3-15e3-43af-9baf-9c12cac734ac service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:32:19.492584 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511330a3-15e3-43af-9baf-9c12cac734ac service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:32:19.492863 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511330a3-15e3-43af-9baf-9c12cac734ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:32:19.501917 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511330a3-15e3-43af-9baf-9c12cac734ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:32:19.503781 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511330a3-15e3-43af-9baf-9c12cac734ac service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:32:19.514464 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 734/1481] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:32:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:32:21.852329 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0880177-9756-423c-8d97-ae7ce64efb38 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:32:21.852892 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0880177-9756-423c-8d97-ae7ce64efb38 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:32:21.852892 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0880177-9756-423c-8d97-ae7ce64efb38 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:32:21.854482 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 748/1482] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:32:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:32:21.859490 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c200f7f-dd7f-4216-bcc7-12b679b6eb57 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:32:21.859951 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c200f7f-dd7f-4216-bcc7-12b679b6eb57 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:32:21.860095 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c200f7f-dd7f-4216-bcc7-12b679b6eb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:32:21.878535 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-9c200f7f-dd7f-4216-bcc7-12b679b6eb57 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:32:21.879720 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-9c200f7f-dd7f-4216-bcc7-12b679b6eb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:32:21.887812 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 735/1483] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:32:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:32:21.923508 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0880177-9756-423c-8d97-ae7ce64efb38 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:32:21.935648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0880177-9756-423c-8d97-ae7ce64efb38 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:32:21.939247 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5584c90d-597e-4314-b2a9-c42766276ec9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:32:21.939898 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584c90d-597e-4314-b2a9-c42766276ec9 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:32:21.940185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584c90d-597e-4314-b2a9-c42766276ec9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:32:21.940468 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584c90d-597e-4314-b2a9-c42766276ec9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:32:21.950000 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5584c90d-597e-4314-b2a9-c42766276ec9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:32:21.951484 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5584c90d-597e-4314-b2a9-c42766276ec9 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:32:21.962793 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 749/1484] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:32:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:32:24.185397 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9ad5d-1061-4ecc-a041-55f81aaeb745 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:32:24.185939 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9ad5d-1061-4ecc-a041-55f81aaeb745 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:32:24.185939 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9ad5d-1061-4ecc-a041-55f81aaeb745 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:32:24.187356 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 736/1485] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:32:24.192468 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e771bcd-c6c9-4877-8701-5a882d079600 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:32:24.192867 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e771bcd-c6c9-4877-8701-5a882d079600 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:32:24.193029 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e771bcd-c6c9-4877-8701-5a882d079600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:32:24.210567 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-4e771bcd-c6c9-4877-8701-5a882d079600 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:32:24.212422 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-4e771bcd-c6c9-4877-8701-5a882d079600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:32:24.220263 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 750/1486] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:32:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:32:24.257944 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c9ad5d-1061-4ecc-a041-55f81aaeb745 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:32:24.268599 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c9ad5d-1061-4ecc-a041-55f81aaeb745 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:32:24.272157 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7ce3b1-2968-48e6-ae35-0d8740bb6672 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:32:24.273766 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ce3b1-2968-48e6-ae35-0d8740bb6672 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:32:24.274053 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ce3b1-2968-48e6-ae35-0d8740bb6672 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:32:24.274905 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ce3b1-2968-48e6-ae35-0d8740bb6672 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:32:24.287115 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7ce3b1-2968-48e6-ae35-0d8740bb6672 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:32:24.288533 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7ce3b1-2968-48e6-ae35-0d8740bb6672 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:32:24.299737 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 737/1487] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:32:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:32:26.503871 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c70ab-edec-4a84-b468-0d6577632c60 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:32:26.504470 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c70ab-edec-4a84-b468-0d6577632c60 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:32:26.504470 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c70ab-edec-4a84-b468-0d6577632c60 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:32:26.505813 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 751/1488] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:32:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:32:26.511858 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2c3a47-a268-405b-86d6-556de12861ab None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:32:26.512317 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2c3a47-a268-405b-86d6-556de12861ab None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:32:26.512416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2c3a47-a268-405b-86d6-556de12861ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:32:26.532650 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-6c2c3a47-a268-405b-86d6-556de12861ab None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:32:26.533851 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6c2c3a47-a268-405b-86d6-556de12861ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:32:26.541651 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 738/1489] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:32:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:32:26.581755 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c70ab-edec-4a84-b468-0d6577632c60 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:32:26.592291 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c70ab-edec-4a84-b468-0d6577632c60 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:32:26.595689 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9364afe-169f-4d65-8102-5be3a659deaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:32:26.596316 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9364afe-169f-4d65-8102-5be3a659deaf service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:32:26.596597 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9364afe-169f-4d65-8102-5be3a659deaf service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:32:26.596867 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9364afe-169f-4d65-8102-5be3a659deaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:32:26.605639 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9364afe-169f-4d65-8102-5be3a659deaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:32:26.607105 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9364afe-169f-4d65-8102-5be3a659deaf service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:32:26.616551 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 752/1490] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:32:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:32:28.880922 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76805d3-fd36-470b-a6bc-ad264b0b82fe None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:32:28.881481 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76805d3-fd36-470b-a6bc-ad264b0b82fe None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:32:28.881481 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76805d3-fd36-470b-a6bc-ad264b0b82fe None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:32:28.882879 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 739/1491] 10.222.0.15 () {54 vars in 926 bytes} [Wed Jun 15 19:32:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:32:28.888533 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43181a76-1fb5-49f7-8d52-01c3985d4430 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:32:28.888958 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43181a76-1fb5-49f7-8d52-01c3985d4430 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:32:28.889073 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43181a76-1fb5-49f7-8d52-01c3985d4430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:32:28.906367 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-43181a76-1fb5-49f7-8d52-01c3985d4430 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:32:28.907626 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-43181a76-1fb5-49f7-8d52-01c3985d4430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:32:28.915933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 753/1492] 10.222.0.15 () {58 vars in 1023 bytes} [Wed Jun 15 19:32:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:32:28.953532 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76805d3-fd36-470b-a6bc-ad264b0b82fe None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:32:28.964172 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76805d3-fd36-470b-a6bc-ad264b0b82fe None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:32:28.967683 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c1c4a7-9755-40d5-83ec-68e9e102a6b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:32:28.968314 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1c4a7-9755-40d5-83ec-68e9e102a6b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:32:28.968596 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1c4a7-9755-40d5-83ec-68e9e102a6b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:32:28.968871 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1c4a7-9755-40d5-83ec-68e9e102a6b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:32:28.978182 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c1c4a7-9755-40d5-83ec-68e9e102a6b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:32:28.979599 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c1c4a7-9755-40d5-83ec-68e9e102a6b4 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:32:28.988807 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 740/1493] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:32:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:32:38.901559 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=33 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:32:38.984795 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=33 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:33:58.977948 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df369d-39aa-4f06-b61e-8cc1a6f549de None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:33:58.978605 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df369d-39aa-4f06-b61e-8cc1a6f549de None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:33:58.978605 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df369d-39aa-4f06-b61e-8cc1a6f549de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:33:58.996019 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-91df369d-39aa-4f06-b61e-8cc1a6f549de None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:33:58.997222 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-91df369d-39aa-4f06-b61e-8cc1a6f549de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:33:59.005304 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 754/1494] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:33:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:33:59.014913 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e4cb4-30d8-4dc1-a446-3a4a682a2912 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:33:59.015401 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e4cb4-30d8-4dc1-a446-3a4a682a2912 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:33:59.015514 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e4cb4-30d8-4dc1-a446-3a4a682a2912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:33:59.033493 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-3d5e4cb4-30d8-4dc1-a446-3a4a682a2912 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:33:59.034730 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3d5e4cb4-30d8-4dc1-a446-3a4a682a2912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:33:59.042467 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 741/1495] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:33:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:33:59.047713 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df369d-39aa-4f06-b61e-8cc1a6f549de None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:33:59.057574 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df369d-39aa-4f06-b61e-8cc1a6f549de None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:33:59.062172 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4ff2e-e1d9-4625-99ae-22ec8e932ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:33:59.062821 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4ff2e-e1d9-4625-99ae-22ec8e932ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:33:59.063099 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4ff2e-e1d9-4625-99ae-22ec8e932ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:33:59.063372 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4ff2e-e1d9-4625-99ae-22ec8e932ee2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:33:59.064123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab4ff2e-e1d9-4625-99ae-22ec8e932ee2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:33:59.064440 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab4ff2e-e1d9-4625-99ae-22ec8e932ee2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:33:59.066071 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: WARNING py.warnings [None req-cab4ff2e-e1d9-4625-99ae-22ec8e932ee2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 15 19:33:59.066071 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: warnings.warn(msg) Jun 15 19:33:59.066071 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: Jun 15 19:33:59.066964 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab4ff2e-e1d9-4625-99ae-22ec8e932ee2 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:33:59.078973 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 755/1496] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:33:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:33:59.087722 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e4cb4-30d8-4dc1-a446-3a4a682a2912 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:33:59.096905 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e4cb4-30d8-4dc1-a446-3a4a682a2912 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:33:59.101385 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2963c32-df96-4ba0-9595-07a6824ea3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:33:59.102025 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2963c32-df96-4ba0-9595-07a6824ea3de admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:33:59.102340 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2963c32-df96-4ba0-9595-07a6824ea3de admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:33:59.102618 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2963c32-df96-4ba0-9595-07a6824ea3de admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:33:59.103317 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2963c32-df96-4ba0-9595-07a6824ea3de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:33:59.105034 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: WARNING py.warnings [None req-a2963c32-df96-4ba0-9595-07a6824ea3de admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 19:33:59.105034 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: warnings.warn(msg) Jun 15 19:33:59.105034 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: Jun 15 19:33:59.106262 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2963c32-df96-4ba0-9595-07a6824ea3de admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:33:59.141787 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 742/1497] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:33:59 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:33:59.146200 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4ff2e-e1d9-4625-99ae-22ec8e932ee2 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:33:59.157550 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4ff2e-e1d9-4625-99ae-22ec8e932ee2 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:33:59.214479 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b384a-db47-4677-bd50-d5e2a3597516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:33:59.215151 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b384a-db47-4677-bd50-d5e2a3597516 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:33:59.215432 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b384a-db47-4677-bd50-d5e2a3597516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:33:59.215734 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b384a-db47-4677-bd50-d5e2a3597516 admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:33:59.216431 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b384a-db47-4677-bd50-d5e2a3597516 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:33:59.217552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: WARNING py.warnings [None req-a06b384a-db47-4677-bd50-d5e2a3597516 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 19:33:59.217552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: warnings.warn(msg) Jun 15 19:33:59.217552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: Jun 15 19:33:59.218784 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b384a-db47-4677-bd50-d5e2a3597516 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:33:59.238935 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 756/1498] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:33:59 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:33:59.243649 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2963c32-df96-4ba0-9595-07a6824ea3de admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:33:59.254058 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2963c32-df96-4ba0-9595-07a6824ea3de admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:33:59.259283 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca76277b-2a51-43d7-8594-94be966b7067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:33:59.259950 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76277b-2a51-43d7-8594-94be966b7067 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:33:59.260259 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76277b-2a51-43d7-8594-94be966b7067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:33:59.260780 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76277b-2a51-43d7-8594-94be966b7067 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:33:59.262077 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca76277b-2a51-43d7-8594-94be966b7067 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:33:59.264007 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: WARNING py.warnings [None req-ca76277b-2a51-43d7-8594-94be966b7067 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 15 19:33:59.264007 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: warnings.warn(msg) Jun 15 19:33:59.264007 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: Jun 15 19:33:59.265667 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca76277b-2a51-43d7-8594-94be966b7067 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:33:59.280462 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 743/1499] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:33:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:33:59.283892 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b384a-db47-4677-bd50-d5e2a3597516 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:33:59.294721 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b384a-db47-4677-bd50-d5e2a3597516 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:33:59.299239 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fb5f95-94e5-4ed8-a040-6dbefa6ab486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:33:59.299902 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb5f95-94e5-4ed8-a040-6dbefa6ab486 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:33:59.300189 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb5f95-94e5-4ed8-a040-6dbefa6ab486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:33:59.300468 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb5f95-94e5-4ed8-a040-6dbefa6ab486 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:33:59.301196 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fb5f95-94e5-4ed8-a040-6dbefa6ab486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:33:59.302234 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: WARNING py.warnings [None req-66fb5f95-94e5-4ed8-a040-6dbefa6ab486 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 15 19:33:59.302234 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: warnings.warn(msg) Jun 15 19:33:59.302234 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: Jun 15 19:33:59.303114 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fb5f95-94e5-4ed8-a040-6dbefa6ab486 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:33:59.317570 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 757/1500] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:33:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:33:59.322371 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca76277b-2a51-43d7-8594-94be966b7067 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:33:59.334322 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca76277b-2a51-43d7-8594-94be966b7067 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:33:59.339084 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1de19e-132b-49cd-963e-2b5c7ce0816e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:33:59.339970 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1de19e-132b-49cd-963e-2b5c7ce0816e admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:33:59.340252 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1de19e-132b-49cd-963e-2b5c7ce0816e admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:33:59.340525 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1de19e-132b-49cd-963e-2b5c7ce0816e admin admin] PATH_INFO: `/v3/projects/a41a3cf912c74318b8dc14e30de43bf1/users/900cc27d069941e79c5538153dd1208a/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:33:59.359564 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1de19e-132b-49cd-963e-2b5c7ce0816e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a41a3cf912c74318b8dc14e30de43bf1, user_id=900cc27d069941e79c5538153dd1208a, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:33:59.360731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: WARNING py.warnings [None req-ed1de19e-132b-49cd-963e-2b5c7ce0816e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 19:33:59.360731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: warnings.warn(msg) Jun 15 19:33:59.360731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: Jun 15 19:33:59.363363 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1de19e-132b-49cd-963e-2b5c7ce0816e admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:33:59.378365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 744/1501] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:33:59 2022] PUT /identity/v3/projects/a41a3cf912c74318b8dc14e30de43bf1/users/900cc27d069941e79c5538153dd1208a/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:33:59.390283 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac131514-1ade-48e2-821d-9c69411743d7 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:33:59.390576 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac131514-1ade-48e2-821d-9c69411743d7 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:33:59.390854 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac131514-1ade-48e2-821d-9c69411743d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:33:59.417573 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-ac131514-1ade-48e2-821d-9c69411743d7 None None] MFA Rules not processed for user `900cc27d069941e79c5538153dd1208a`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:33:59.418823 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-ac131514-1ade-48e2-821d-9c69411743d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:33:59.484057 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 758/1502] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:33:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:33:59.497860 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa5251-4896-40f8-bad3-52e0db5fe2b5 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:33:59.498254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa5251-4896-40f8-bad3-52e0db5fe2b5 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:33:59.498540 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa5251-4896-40f8-bad3-52e0db5fe2b5 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:33:59.499955 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 745/1503] 10.222.0.15 () {54 vars in 963 bytes} [Wed Jun 15 19:33:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:33:59.506649 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184542e-cdd5-4d72-878e-9f26f40161eb None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:33:59.507765 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184542e-cdd5-4d72-878e-9f26f40161eb None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:33:59.508308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184542e-cdd5-4d72-878e-9f26f40161eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:33:59.527604 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-9184542e-cdd5-4d72-878e-9f26f40161eb None None] MFA Rules not processed for user `ec5882def40a4e0d8a792eecc12440a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:33:59.529015 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-9184542e-cdd5-4d72-878e-9f26f40161eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:33:59.575600 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:33:59.583975 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 759/1504] 10.222.0.15 () {58 vars in 1060 bytes} [Wed Jun 15 19:33:59 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:33:59.591914 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa5251-4896-40f8-bad3-52e0db5fe2b5 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:33:59.602496 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa5251-4896-40f8-bad3-52e0db5fe2b5 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:33:59.606242 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3969c8-7d1c-4dab-b440-8b79d37e14df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:33:59.606876 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3969c8-7d1c-4dab-b440-8b79d37e14df service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:33:59.607157 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3969c8-7d1c-4dab-b440-8b79d37e14df service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:33:59.607432 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3969c8-7d1c-4dab-b440-8b79d37e14df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:33:59.618668 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3969c8-7d1c-4dab-b440-8b79d37e14df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:33:59.620099 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3969c8-7d1c-4dab-b440-8b79d37e14df service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:33:59.629405 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 746/1505] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:33:59 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:33:59.674961 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89beb68-afe8-4b88-ac5c-4f1a507eb6ee None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:33:59.675370 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89beb68-afe8-4b88-ac5c-4f1a507eb6ee None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:33:59.675657 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89beb68-afe8-4b88-ac5c-4f1a507eb6ee None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:33:59.676953 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 760/1506] 10.222.0.15 () {52 vars in 829 bytes} [Wed Jun 15 19:33:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:33:59.681037 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3969c8-7d1c-4dab-b440-8b79d37e14df service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:33:59.743155 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3969c8-7d1c-4dab-b440-8b79d37e14df service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:33:59.747656 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc462ba-e8c7-4864-af62-6e96fe4131e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:33:59.748530 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc462ba-e8c7-4864-af62-6e96fe4131e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:33:59.748819 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc462ba-e8c7-4864-af62-6e96fe4131e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:33:59.749103 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc462ba-e8c7-4864-af62-6e96fe4131e3 admin admin] PATH_INFO: `/v3/users/900cc27d069941e79c5538153dd1208a` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:33:59.750466 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc462ba-e8c7-4864-af62-6e96fe4131e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=900cc27d069941e79c5538153dd1208a)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:33:59.751493 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: WARNING py.warnings [None req-3bc462ba-e8c7-4864-af62-6e96fe4131e3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 19:33:59.751493 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: warnings.warn(msg) Jun 15 19:33:59.751493 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: Jun 15 19:33:59.752700 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc462ba-e8c7-4864-af62-6e96fe4131e3 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:33:59.820798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-3bc462ba-e8c7-4864-af62-6e96fe4131e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '900cc27d069941e79c5538153dd1208a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:33:59.821522 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-3bc462ba-e8c7-4864-af62-6e96fe4131e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '900cc27d069941e79c5538153dd1208a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:33:59.822266 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-3bc462ba-e8c7-4864-af62-6e96fe4131e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '900cc27d069941e79c5538153dd1208a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:33:59.840505 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-3bc462ba-e8c7-4864-af62-6e96fe4131e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '900cc27d069941e79c5538153dd1208a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:33:59.853407 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-3bc462ba-e8c7-4864-af62-6e96fe4131e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '900cc27d069941e79c5538153dd1208a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:33:59.857378 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:33:59.871534 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 747/1507] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:33:59 2022] DELETE /identity/v3/users/900cc27d069941e79c5538153dd1208a => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:33:59.876047 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65e6fb-7519-41f6-af09-20afd5f90a0d None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:33:59.876543 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65e6fb-7519-41f6-af09-20afd5f90a0d None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:33:59.877039 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65e6fb-7519-41f6-af09-20afd5f90a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:33:59.896474 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-ae65e6fb-7519-41f6-af09-20afd5f90a0d None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:33:59.897673 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-ae65e6fb-7519-41f6-af09-20afd5f90a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:33:59.954437 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 761/1508] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:33:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:33:59.963953 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc462ba-e8c7-4864-af62-6e96fe4131e3 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:33:59.965363 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3bc462ba-e8c7-4864-af62-6e96fe4131e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:34:00.021254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc462ba-e8c7-4864-af62-6e96fe4131e3 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:34:00.025467 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f5df7c-d428-474c-8421-fa9737965e91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:34:00.026116 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5df7c-d428-474c-8421-fa9737965e91 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:34:00.026405 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5df7c-d428-474c-8421-fa9737965e91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:34:00.026678 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5df7c-d428-474c-8421-fa9737965e91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:34:00.035423 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f5df7c-d428-474c-8421-fa9737965e91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:34:00.036850 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f5df7c-d428-474c-8421-fa9737965e91 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:34:00.046690 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 748/1509] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:33:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:34:00.370368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae65e6fb-7519-41f6-af09-20afd5f90a0d None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:34:00.371836 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-ae65e6fb-7519-41f6-af09-20afd5f90a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:34:00.385040 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae65e6fb-7519-41f6-af09-20afd5f90a0d None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:34:00.390475 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b91257-e054-4635-8092-72e6c45210bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:34:00.391509 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b91257-e054-4635-8092-72e6c45210bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:34:00.391796 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b91257-e054-4635-8092-72e6c45210bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:34:00.392070 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b91257-e054-4635-8092-72e6c45210bc admin admin] PATH_INFO: `/v3/projects/a41a3cf912c74318b8dc14e30de43bf1` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:34:00.393278 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b91257-e054-4635-8092-72e6c45210bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a41a3cf912c74318b8dc14e30de43bf1)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:34:00.394985 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: WARNING py.warnings [None req-a9b91257-e054-4635-8092-72e6c45210bc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 19:34:00.394985 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: warnings.warn(msg) Jun 15 19:34:00.394985 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: Jun 15 19:34:00.396376 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b91257-e054-4635-8092-72e6c45210bc admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:34:00.457351 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-a9b91257-e054-4635-8092-72e6c45210bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a41a3cf912c74318b8dc14e30de43bf1', 'request_id': 'req-a9b91257-e054-4635-8092-72e6c45210bc'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:34:00.467948 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-a9b91257-e054-4635-8092-72e6c45210bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a41a3cf912c74318b8dc14e30de43bf1', 'request_id': 'req-a9b91257-e054-4635-8092-72e6c45210bc'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:34:00.481736 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-a9b91257-e054-4635-8092-72e6c45210bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a41a3cf912c74318b8dc14e30de43bf1', 'request_id': 'req-a9b91257-e054-4635-8092-72e6c45210bc'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:34:00.493002 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-a9b91257-e054-4635-8092-72e6c45210bc admin admin] The token cache is being invalidate because project a41a3cf912c74318b8dc14e30de43bf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:34:00.493286 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-a9b91257-e054-4635-8092-72e6c45210bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:34:00.494023 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-a9b91257-e054-4635-8092-72e6c45210bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:34:00.495607 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 762/1510] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:34:00 2022] DELETE /identity/v3/projects/a41a3cf912c74318b8dc14e30de43bf1 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:34:10.042525 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:34:10.489949 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:35:42.622327 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4ed40-4bab-443e-8189-85767b215ecf None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:35:42.622893 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4ed40-4bab-443e-8189-85767b215ecf None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:35:42.622893 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4ed40-4bab-443e-8189-85767b215ecf None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:35:42.624194 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 749/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 19:35:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:35:42.629625 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c46f1-bf9a-4af5-a706-dfc580c9deee None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:35:42.630074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c46f1-bf9a-4af5-a706-dfc580c9deee None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:35:42.630213 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c46f1-bf9a-4af5-a706-dfc580c9deee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:35:42.648775 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-c78c46f1-bf9a-4af5-a706-dfc580c9deee None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:35:42.649973 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-c78c46f1-bf9a-4af5-a706-dfc580c9deee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:35:42.707663 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 763/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 19:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:35:42.876875 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4ed40-4bab-443e-8189-85767b215ecf None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:35:42.879509 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-83d4ed40-4bab-443e-8189-85767b215ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:35:42.952450 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4ed40-4bab-443e-8189-85767b215ecf None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:35:42.958001 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74265a5a-d6b6-4e0b-aa6a-248ee9f81fca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:35:42.958692 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74265a5a-d6b6-4e0b-aa6a-248ee9f81fca service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:35:42.958995 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74265a5a-d6b6-4e0b-aa6a-248ee9f81fca service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:35:42.959287 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74265a5a-d6b6-4e0b-aa6a-248ee9f81fca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:35:42.967498 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74265a5a-d6b6-4e0b-aa6a-248ee9f81fca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:35:42.969144 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74265a5a-d6b6-4e0b-aa6a-248ee9f81fca service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:35:42.978600 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 750/1513] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:35:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:35:43.848738 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc8e10-c413-4f3c-9a9c-ea6208cbbc37 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:35:43.849258 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc8e10-c413-4f3c-9a9c-ea6208cbbc37 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:35:43.849358 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc8e10-c413-4f3c-9a9c-ea6208cbbc37 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:35:43.850811 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 764/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 19:35:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:35:43.855712 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486da2a-0d87-4a77-9dcf-66ea30f67b01 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:35:43.856133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486da2a-0d87-4a77-9dcf-66ea30f67b01 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:35:43.856293 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486da2a-0d87-4a77-9dcf-66ea30f67b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:35:43.874452 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-3486da2a-0d87-4a77-9dcf-66ea30f67b01 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:35:43.875648 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3486da2a-0d87-4a77-9dcf-66ea30f67b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:35:43.884318 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 751/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 19:35:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:35:44.043605 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc8e10-c413-4f3c-9a9c-ea6208cbbc37 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:35:44.051769 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c63fe-0260-4c2e-97ca-8d83d13761cf None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:35:44.052141 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c63fe-0260-4c2e-97ca-8d83d13761cf None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:35:44.052485 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c63fe-0260-4c2e-97ca-8d83d13761cf None None] PATH_INFO: `/v3` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:35:44.053864 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 752/1516] 10.222.0.42 () {54 vars in 925 bytes} [Wed Jun 15 19:35:44 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:35:44.057890 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc8e10-c413-4f3c-9a9c-ea6208cbbc37 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:35:44.059703 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af2ecb6-cd0f-4d40-9d65-b424c284a958 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:35:44.059937 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af2ecb6-cd0f-4d40-9d65-b424c284a958 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:35:44.060213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af2ecb6-cd0f-4d40-9d65-b424c284a958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:35:44.063410 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218043fb-f277-4067-acc5-a7d1f42a33ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:35:44.064041 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218043fb-f277-4067-acc5-a7d1f42a33ca service nova] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:35:44.064321 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218043fb-f277-4067-acc5-a7d1f42a33ca service nova] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:35:44.064594 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218043fb-f277-4067-acc5-a7d1f42a33ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:35:44.074589 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218043fb-f277-4067-acc5-a7d1f42a33ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:35:44.074950 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218043fb-f277-4067-acc5-a7d1f42a33ca service nova] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:35:44.078554 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-3af2ecb6-cd0f-4d40-9d65-b424c284a958 None None] MFA Rules not processed for user `cb0ec2ef6c4748e181f1963e7dfe30b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:35:44.079731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3af2ecb6-cd0f-4d40-9d65-b424c284a958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:35:44.085002 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 765/1517] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:35:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:35:44.147332 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 753/1518] 10.222.0.42 () {58 vars in 1015 bytes} [Wed Jun 15 19:35:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:35:47.922324 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300af34f-e48e-4eba-bfa1-026569da7798 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:35:47.922846 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300af34f-e48e-4eba-bfa1-026569da7798 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:35:47.922846 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300af34f-e48e-4eba-bfa1-026569da7798 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:35:47.924152 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 766/1519] 10.222.0.15 () {54 vars in 960 bytes} [Wed Jun 15 19:35:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:35:47.929814 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3e2ec-c935-4405-8af7-4534582d6bf7 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:35:47.930325 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3e2ec-c935-4405-8af7-4534582d6bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:35:47.930433 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3e2ec-c935-4405-8af7-4534582d6bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:35:47.948234 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c2b3e2ec-c935-4405-8af7-4534582d6bf7 None None] MFA Rules not processed for user `cb0ec2ef6c4748e181f1963e7dfe30b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:35:47.949421 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c2b3e2ec-c935-4405-8af7-4534582d6bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:35:47.957802 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 754/1520] 10.222.0.15 () {58 vars in 1057 bytes} [Wed Jun 15 19:35:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:35:47.965249 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300af34f-e48e-4eba-bfa1-026569da7798 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:35:47.978729 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300af34f-e48e-4eba-bfa1-026569da7798 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:35:47.984530 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce7b74-c948-4461-9d88-de2965695309 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb0ec2ef6c4748e181f1963e7dfe30b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:35:47.985233 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce7b74-c948-4461-9d88-de2965695309 service placement] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:35:47.985517 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce7b74-c948-4461-9d88-de2965695309 service placement] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:35:47.985792 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce7b74-c948-4461-9d88-de2965695309 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:35:47.994456 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce7b74-c948-4461-9d88-de2965695309 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:35:47.996051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce7b74-c948-4461-9d88-de2965695309 service placement] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:35:48.005661 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 767/1521] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:35:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:35:55.339832 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600275f-5a9f-4d01-a6e7-5a8871dfd7b2 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:35:55.340364 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600275f-5a9f-4d01-a6e7-5a8871dfd7b2 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:35:55.340364 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600275f-5a9f-4d01-a6e7-5a8871dfd7b2 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:35:55.341814 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 755/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 19:35:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:35:55.346604 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924a57a-731a-42de-95b8-abb602de1c9c None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:35:55.347014 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924a57a-731a-42de-95b8-abb602de1c9c None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:35:55.347163 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924a57a-731a-42de-95b8-abb602de1c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:35:55.364955 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-6924a57a-731a-42de-95b8-abb602de1c9c None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:35:55.366207 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-6924a57a-731a-42de-95b8-abb602de1c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:35:55.374213 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 768/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 19:35:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:35:55.529553 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600275f-5a9f-4d01-a6e7-5a8871dfd7b2 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:35:55.544420 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600275f-5a9f-4d01-a6e7-5a8871dfd7b2 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:35:55.549913 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd2d22-75bd-4cb4-a67e-2442ca5f1d30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:35:55.550583 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd2d22-75bd-4cb4-a67e-2442ca5f1d30 service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:35:55.550869 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd2d22-75bd-4cb4-a67e-2442ca5f1d30 service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:35:55.551145 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd2d22-75bd-4cb4-a67e-2442ca5f1d30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:35:55.559222 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd2d22-75bd-4cb4-a67e-2442ca5f1d30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:35:55.560812 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd2d22-75bd-4cb4-a67e-2442ca5f1d30 service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:35:55.570194 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 756/1524] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:35:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:35:56.433216 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846076d8-6844-4251-a777-378a3eb17d38 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:35:56.433766 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846076d8-6844-4251-a777-378a3eb17d38 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:35:56.433868 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846076d8-6844-4251-a777-378a3eb17d38 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:35:56.435235 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 769/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 19:35:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:35:56.440240 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e4e0a-3fcc-41b4-9fbc-6aa27161a1f2 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:35:56.441235 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e4e0a-3fcc-41b4-9fbc-6aa27161a1f2 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:35:56.441407 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e4e0a-3fcc-41b4-9fbc-6aa27161a1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:35:56.459417 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-423e4e0a-3fcc-41b4-9fbc-6aa27161a1f2 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:35:56.460599 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-423e4e0a-3fcc-41b4-9fbc-6aa27161a1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:35:56.468350 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 757/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 19:35:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:35:56.616544 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846076d8-6844-4251-a777-378a3eb17d38 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:35:56.631365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846076d8-6844-4251-a777-378a3eb17d38 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:35:56.636637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66096c36-47b9-45c6-a3eb-8787c87cffa8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:35:56.637267 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66096c36-47b9-45c6-a3eb-8787c87cffa8 service nova] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:35:56.637554 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66096c36-47b9-45c6-a3eb-8787c87cffa8 service nova] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:35:56.637832 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66096c36-47b9-45c6-a3eb-8787c87cffa8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:35:56.646152 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66096c36-47b9-45c6-a3eb-8787c87cffa8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:35:56.647736 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66096c36-47b9-45c6-a3eb-8787c87cffa8 service nova] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:35:56.656844 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 770/1527] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:35:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:36:00.652344 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df0d9c-7119-4eb4-a582-33f398d27799 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:36:00.652882 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df0d9c-7119-4eb4-a582-33f398d27799 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:36:00.652882 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df0d9c-7119-4eb4-a582-33f398d27799 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:36:00.654269 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 758/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 19:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:36:00.659342 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725380b2-6d10-449d-abea-037e50bce8fc None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:36:00.659770 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725380b2-6d10-449d-abea-037e50bce8fc None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:36:00.659885 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725380b2-6d10-449d-abea-037e50bce8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:36:00.677310 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-725380b2-6d10-449d-abea-037e50bce8fc None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:36:00.678529 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-725380b2-6d10-449d-abea-037e50bce8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:36:00.686203 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 771/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 19:36:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:36:00.722902 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74df0d9c-7119-4eb4-a582-33f398d27799 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:36:00.724572 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-74df0d9c-7119-4eb4-a582-33f398d27799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:36:00.779025 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:36:00.783504 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74df0d9c-7119-4eb4-a582-33f398d27799 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:36:00.786996 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427ac6a-3b32-4b58-962e-181db6388c6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:36:00.787658 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427ac6a-3b32-4b58-962e-181db6388c6a service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:36:00.787940 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427ac6a-3b32-4b58-962e-181db6388c6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:36:00.788211 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427ac6a-3b32-4b58-962e-181db6388c6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:36:00.796915 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4427ac6a-3b32-4b58-962e-181db6388c6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:36:00.798665 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4427ac6a-3b32-4b58-962e-181db6388c6a service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:36:00.808295 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 759/1530] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:36:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:36:02.907470 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b887f-a4de-4010-8f85-6dba47ff7ae3 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:36:02.908058 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b887f-a4de-4010-8f85-6dba47ff7ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:36:02.908058 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b887f-a4de-4010-8f85-6dba47ff7ae3 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:36:02.909932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 772/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 19:36:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:36:02.915007 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f3be5f-54f8-4a5a-b081-620b1e36f493 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:36:02.915388 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f3be5f-54f8-4a5a-b081-620b1e36f493 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:36:02.915568 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f3be5f-54f8-4a5a-b081-620b1e36f493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:36:02.933213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-d2f3be5f-54f8-4a5a-b081-620b1e36f493 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:36:02.934457 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d2f3be5f-54f8-4a5a-b081-620b1e36f493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:36:02.942309 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 760/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 19:36:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:36:02.976767 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b887f-a4de-4010-8f85-6dba47ff7ae3 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:36:02.988086 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b887f-a4de-4010-8f85-6dba47ff7ae3 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:36:02.991649 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3678efc-f313-4868-89ea-6d4d97237683 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:36:02.992288 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3678efc-f313-4868-89ea-6d4d97237683 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:36:02.992574 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3678efc-f313-4868-89ea-6d4d97237683 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:36:02.992854 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3678efc-f313-4868-89ea-6d4d97237683 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:36:03.001715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3678efc-f313-4868-89ea-6d4d97237683 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:36:03.003478 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3678efc-f313-4868-89ea-6d4d97237683 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:36:03.012790 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 773/1533] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:36:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:36:06.676073 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360ea2a-eef3-46a2-ac7a-15e69b7fc9d3 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:36:06.676579 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360ea2a-eef3-46a2-ac7a-15e69b7fc9d3 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:36:06.676579 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360ea2a-eef3-46a2-ac7a-15e69b7fc9d3 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:36:06.677948 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 761/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 19:36:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:36:06.682956 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc9eef7-8865-4c00-849d-490f624b50a5 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:36:06.683366 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc9eef7-8865-4c00-849d-490f624b50a5 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:36:06.683503 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc9eef7-8865-4c00-849d-490f624b50a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:36:06.701502 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-6fc9eef7-8865-4c00-849d-490f624b50a5 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:36:06.702733 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-6fc9eef7-8865-4c00-849d-490f624b50a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:36:06.711038 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 774/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 19:36:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:36:06.975260 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1360ea2a-eef3-46a2-ac7a-15e69b7fc9d3 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:36:06.976970 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1360ea2a-eef3-46a2-ac7a-15e69b7fc9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:36:07.033429 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1360ea2a-eef3-46a2-ac7a-15e69b7fc9d3 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:36:07.037507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b1e7a-9b9e-4dd9-9d80-11370138dc56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:36:07.038183 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b1e7a-9b9e-4dd9-9d80-11370138dc56 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:36:07.038473 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b1e7a-9b9e-4dd9-9d80-11370138dc56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:36:07.038751 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b1e7a-9b9e-4dd9-9d80-11370138dc56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:36:07.047679 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b1e7a-9b9e-4dd9-9d80-11370138dc56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:36:07.049111 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b1e7a-9b9e-4dd9-9d80-11370138dc56 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:36:07.058314 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 762/1536] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:36:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:36:09.206422 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e962b-56c1-480c-bd6d-f968c9e59fcd None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:36:09.206946 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e962b-56c1-480c-bd6d-f968c9e59fcd None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:36:09.206946 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e962b-56c1-480c-bd6d-f968c9e59fcd None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:36:09.208269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 775/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 19:36:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:36:09.213538 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89147048-e0e8-445c-85c5-804e211091d3 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:36:09.213954 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89147048-e0e8-445c-85c5-804e211091d3 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:36:09.214128 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89147048-e0e8-445c-85c5-804e211091d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:36:09.232458 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-89147048-e0e8-445c-85c5-804e211091d3 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:36:09.233647 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-89147048-e0e8-445c-85c5-804e211091d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:36:09.241493 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 763/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 19:36:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:36:09.505697 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e962b-56c1-480c-bd6d-f968c9e59fcd None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:36:09.507415 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-9f1e962b-56c1-480c-bd6d-f968c9e59fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:36:09.520783 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e962b-56c1-480c-bd6d-f968c9e59fcd None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:36:09.524253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c9072-7276-43ef-99d3-2e2cf9320e93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:36:09.524873 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c9072-7276-43ef-99d3-2e2cf9320e93 service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:36:09.525151 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c9072-7276-43ef-99d3-2e2cf9320e93 service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:36:09.525421 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c9072-7276-43ef-99d3-2e2cf9320e93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:36:09.534173 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c9072-7276-43ef-99d3-2e2cf9320e93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:36:09.535585 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c9072-7276-43ef-99d3-2e2cf9320e93 service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:36:09.544762 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 776/1539] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:36:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:36:19.227771 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=36 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:36:19.540657 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=66 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:37:39.538354 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9512ed8-01a0-4606-ad3b-a9a0fea06915 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:39.539015 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9512ed8-01a0-4606-ad3b-a9a0fea06915 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:39.539015 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9512ed8-01a0-4606-ad3b-a9a0fea06915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:39.570235 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c9512ed8-01a0-4606-ad3b-a9a0fea06915 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:39.571598 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c9512ed8-01a0-4606-ad3b-a9a0fea06915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:39.581043 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 764/1540] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:37:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:37:39.591173 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfba01-a16e-4af5-80fd-9e7139d85ca2 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:39.591728 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfba01-a16e-4af5-80fd-9e7139d85ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:39.591728 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfba01-a16e-4af5-80fd-9e7139d85ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:39.610593 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8d98a-248c-407b-9a7d-cd15fcd4f016 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:39.610935 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8d98a-248c-407b-9a7d-cd15fcd4f016 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:39.611212 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8d98a-248c-407b-9a7d-cd15fcd4f016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:39.624519 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-00bfba01-a16e-4af5-80fd-9e7139d85ca2 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:39.625722 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-00bfba01-a16e-4af5-80fd-9e7139d85ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:39.642284 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 777/1541] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:37:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:37:39.643183 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-3ad8d98a-248c-407b-9a7d-cd15fcd4f016 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:39.644630 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3ad8d98a-248c-407b-9a7d-cd15fcd4f016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:39.647186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bfba01-a16e-4af5-80fd-9e7139d85ca2 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:39.662496 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bfba01-a16e-4af5-80fd-9e7139d85ca2 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:39.662725 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 765/1542] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:37:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:37:39.671814 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138553f9-4e2f-462e-93c2-70b56e1fb7f7 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:39.672168 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138553f9-4e2f-462e-93c2-70b56e1fb7f7 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:39.672371 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138553f9-4e2f-462e-93c2-70b56e1fb7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:39.672477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f6e0b-d28e-4d70-a52b-db1c67bc04df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:39.673074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f6e0b-d28e-4d70-a52b-db1c67bc04df admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:39.673359 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f6e0b-d28e-4d70-a52b-db1c67bc04df admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:39.673636 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f6e0b-d28e-4d70-a52b-db1c67bc04df admin admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:39.674409 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786f6e0b-d28e-4d70-a52b-db1c67bc04df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:37:39.674722 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786f6e0b-d28e-4d70-a52b-db1c67bc04df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:39.676410 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786f6e0b-d28e-4d70-a52b-db1c67bc04df admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:39.697773 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-138553f9-4e2f-462e-93c2-70b56e1fb7f7 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:39.699107 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-138553f9-4e2f-462e-93c2-70b56e1fb7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:39.702259 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 778/1543] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:37:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:37:39.703783 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfd047-ec91-4380-8ab6-c34ad9cb6c9b None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:39.704077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfd047-ec91-4380-8ab6-c34ad9cb6c9b None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:39.704355 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfd047-ec91-4380-8ab6-c34ad9cb6c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:39.718260 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 766/1544] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:37:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:37:39.719204 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138553f9-4e2f-462e-93c2-70b56e1fb7f7 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:39.741486 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138553f9-4e2f-462e-93c2-70b56e1fb7f7 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:39.743317 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-72bfd047-ec91-4380-8ab6-c34ad9cb6c9b None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:39.744992 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-72bfd047-ec91-4380-8ab6-c34ad9cb6c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:39.751996 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb93fca-6a08-4dd7-9e50-4660846cca29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:39.752708 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb93fca-6a08-4dd7-9e50-4660846cca29 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:39.752991 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb93fca-6a08-4dd7-9e50-4660846cca29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:39.753264 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb93fca-6a08-4dd7-9e50-4660846cca29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:39.754003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb93fca-6a08-4dd7-9e50-4660846cca29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:39.756099 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb93fca-6a08-4dd7-9e50-4660846cca29 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:39.784078 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 779/1545] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:37:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:37:39.785083 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bfd047-ec91-4380-8ab6-c34ad9cb6c9b None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:39.800479 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bfd047-ec91-4380-8ab6-c34ad9cb6c9b None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:39.808172 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 767/1546] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:37:39 2022] POST /identity/v3/projects => generated 389 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:37:39.810038 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd79f3f-d71d-4422-9419-f2cd05560be8 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:39.810216 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd79f3f-d71d-4422-9419-f2cd05560be8 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:39.810455 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd79f3f-d71d-4422-9419-f2cd05560be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:39.838583 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-ffd79f3f-d71d-4422-9419-f2cd05560be8 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:39.846575 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ffd79f3f-d71d-4422-9419-f2cd05560be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:39.942083 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 768/1547] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:37:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:37:39.942979 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd79f3f-d71d-4422-9419-f2cd05560be8 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:39.956458 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd79f3f-d71d-4422-9419-f2cd05560be8 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:39.972211 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d886c4-a038-4444-9b2d-9b6152c5c5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:39.973188 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d886c4-a038-4444-9b2d-9b6152c5c5ec admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:39.973481 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d886c4-a038-4444-9b2d-9b6152c5c5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:39.973753 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d886c4-a038-4444-9b2d-9b6152c5c5ec admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:39.974488 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d886c4-a038-4444-9b2d-9b6152c5c5ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:39.975532 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: WARNING py.warnings [None req-a8d886c4-a038-4444-9b2d-9b6152c5c5ec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 19:37:39.975532 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: warnings.warn(msg) Jun 15 19:37:39.975532 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: Jun 15 19:37:39.976760 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d886c4-a038-4444-9b2d-9b6152c5c5ec admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:40.014556 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 769/1548] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:37:39 2022] POST /identity/v3/users => generated 295 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:37:40.017443 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b775462-33a3-496a-b58c-e778717e418c None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:40.017699 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b775462-33a3-496a-b58c-e778717e418c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:40.017975 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b775462-33a3-496a-b58c-e778717e418c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:40.051760 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-5b775462-33a3-496a-b58c-e778717e418c None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:40.052889 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-5b775462-33a3-496a-b58c-e778717e418c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:40.054405 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d742e-a9e2-4d44-a930-7c30cdbca55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:40.055125 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d742e-a9e2-4d44-a930-7c30cdbca55a admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:40.055407 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d742e-a9e2-4d44-a930-7c30cdbca55a admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:40.055677 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d742e-a9e2-4d44-a930-7c30cdbca55a admin admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:40.056408 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d742e-a9e2-4d44-a930-7c30cdbca55a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:37:40.056713 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d742e-a9e2-4d44-a930-7c30cdbca55a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:40.058324 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d742e-a9e2-4d44-a930-7c30cdbca55a admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:40.063937 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 770/1549] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:37:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:37:40.064768 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b775462-33a3-496a-b58c-e778717e418c None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:40.070530 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 780/1550] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:37:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:37:40.071352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d742e-a9e2-4d44-a930-7c30cdbca55a admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:40.080943 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b775462-33a3-496a-b58c-e778717e418c None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:40.082203 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d742e-a9e2-4d44-a930-7c30cdbca55a admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:40.085636 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09175334-4c10-4836-973f-b9ec533ea64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:40.086314 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09175334-4c10-4836-973f-b9ec533ea64d admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:40.086602 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09175334-4c10-4836-973f-b9ec533ea64d admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:40.086827 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2619cc64-d4db-4c7b-8d00-4dd3d96bbd9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:40.086973 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09175334-4c10-4836-973f-b9ec533ea64d admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:40.087478 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619cc64-d4db-4c7b-8d00-4dd3d96bbd9f admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:40.087601 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09175334-4c10-4836-973f-b9ec533ea64d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:37:40.087752 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619cc64-d4db-4c7b-8d00-4dd3d96bbd9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:40.087867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09175334-4c10-4836-973f-b9ec533ea64d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:40.088020 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619cc64-d4db-4c7b-8d00-4dd3d96bbd9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:40.088868 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2619cc64-d4db-4c7b-8d00-4dd3d96bbd9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:40.088986 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: WARNING py.warnings [None req-09175334-4c10-4836-973f-b9ec533ea64d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 15 19:37:40.088986 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: warnings.warn(msg) Jun 15 19:37:40.088986 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: Jun 15 19:37:40.089776 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09175334-4c10-4836-973f-b9ec533ea64d admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:40.090404 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2619cc64-d4db-4c7b-8d00-4dd3d96bbd9f admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:40.101006 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 781/1551] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:37:40.102061 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20c3f7-44f1-45cb-a6ee-aac117354b1b None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:40.102374 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20c3f7-44f1-45cb-a6ee-aac117354b1b None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:40.102669 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20c3f7-44f1-45cb-a6ee-aac117354b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:40.106121 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 771/1552] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:37:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:37:40.106121 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3607949-bfa6-440c-82eb-b8f16a679e30 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:40.106121 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3607949-bfa6-440c-82eb-b8f16a679e30 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:40.106431 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3607949-bfa6-440c-82eb-b8f16a679e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:40.122829 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-6c20c3f7-44f1-45cb-a6ee-aac117354b1b None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:40.124079 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-a3607949-bfa6-440c-82eb-b8f16a679e30 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:40.124541 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-6c20c3f7-44f1-45cb-a6ee-aac117354b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:40.125227 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a3607949-bfa6-440c-82eb-b8f16a679e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:40.132482 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 782/1553] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:37:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:37:40.133053 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 772/1554] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:37:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:37:40.133053 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179798a-3744-4be4-9f47-20285f94c1e5 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:40.133053 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179798a-3744-4be4-9f47-20285f94c1e5 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:40.133053 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179798a-3744-4be4-9f47-20285f94c1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:40.142267 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5885c01-edc8-460d-8a2d-d5e6614469dd None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:40.142267 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5885c01-edc8-460d-8a2d-d5e6614469dd None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:40.142267 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5885c01-edc8-460d-8a2d-d5e6614469dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:40.156543 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-a179798a-3744-4be4-9f47-20285f94c1e5 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:40.157747 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a179798a-3744-4be4-9f47-20285f94c1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:40.159122 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-c5885c01-edc8-460d-8a2d-d5e6614469dd None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:40.160258 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-c5885c01-edc8-460d-8a2d-d5e6614469dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:40.165598 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 773/1555] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:37:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:37:40.166883 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b8baed-bbda-4489-b8bb-de2c2c64c6f1 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:40.167178 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b8baed-bbda-4489-b8bb-de2c2c64c6f1 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:40.167456 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b8baed-bbda-4489-b8bb-de2c2c64c6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:40.168594 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 783/1556] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:37:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:37:40.169203 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5885c01-edc8-460d-8a2d-d5e6614469dd None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:40.179606 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5885c01-edc8-460d-8a2d-d5e6614469dd None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:40.184157 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcbbd53-38f9-48e2-af73-2ada006c3a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:40.184803 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbbd53-38f9-48e2-af73-2ada006c3a45 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:40.185086 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbbd53-38f9-48e2-af73-2ada006c3a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:40.185359 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbbd53-38f9-48e2-af73-2ada006c3a45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:40.186648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcbbd53-38f9-48e2-af73-2ada006c3a45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:40.187614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: WARNING py.warnings [None req-1bcbbd53-38f9-48e2-af73-2ada006c3a45 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 19:37:40.187614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: warnings.warn(msg) Jun 15 19:37:40.187614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: Jun 15 19:37:40.187908 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-b8b8baed-bbda-4489-b8bb-de2c2c64c6f1 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:40.188796 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcbbd53-38f9-48e2-af73-2ada006c3a45 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:40.189085 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b8b8baed-bbda-4489-b8bb-de2c2c64c6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:40.197668 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 774/1557] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:37:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:37:40.198510 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b8baed-bbda-4489-b8bb-de2c2c64c6f1 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:40.209447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b8baed-bbda-4489-b8bb-de2c2c64c6f1 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:40.214937 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2fc3d7-bb98-43e5-8458-2c47f3f5b763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:40.215591 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2fc3d7-bb98-43e5-8458-2c47f3f5b763 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:40.215877 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2fc3d7-bb98-43e5-8458-2c47f3f5b763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:40.216147 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2fc3d7-bb98-43e5-8458-2c47f3f5b763 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:40.216869 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2fc3d7-bb98-43e5-8458-2c47f3f5b763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:40.221497 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 784/1558] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:37:40 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:37:40.221667 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2fc3d7-bb98-43e5-8458-2c47f3f5b763 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:40.221787 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcbbd53-38f9-48e2-af73-2ada006c3a45 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:40.231494 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 775/1559] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:37:40.232999 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af1b4b-8b93-4cb4-bd94-5d28e394b689 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:40.233293 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af1b4b-8b93-4cb4-bd94-5d28e394b689 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:40.233702 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af1b4b-8b93-4cb4-bd94-5d28e394b689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:40.252658 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-b5af1b4b-8b93-4cb4-bd94-5d28e394b689 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:40.253866 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b5af1b4b-8b93-4cb4-bd94-5d28e394b689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:40.281020 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcbbd53-38f9-48e2-af73-2ada006c3a45 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:40.285662 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292603f1-720a-4ac8-a447-ad62b003b2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:40.286338 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292603f1-720a-4ac8-a447-ad62b003b2f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:40.286618 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292603f1-720a-4ac8-a447-ad62b003b2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:40.286889 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292603f1-720a-4ac8-a447-ad62b003b2f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:40.287569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292603f1-720a-4ac8-a447-ad62b003b2f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:40.289455 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292603f1-720a-4ac8-a447-ad62b003b2f1 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:40.299591 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:37:40.315403 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 785/1560] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:37:40 2022] POST /identity/v3/projects => generated 377 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:37:40.315965 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 776/1561] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:37:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:37:40.319625 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d6172-ed0f-470d-99b9-47b62aba3f6d None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:40.319733 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f800c-3ff2-42f8-bbd9-c9cfa76d1c9b None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:40.319833 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d6172-ed0f-470d-99b9-47b62aba3f6d None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:40.319833 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d6172-ed0f-470d-99b9-47b62aba3f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:40.320009 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f800c-3ff2-42f8-bbd9-c9cfa76d1c9b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:40.320009 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f800c-3ff2-42f8-bbd9-c9cfa76d1c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:40.330838 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:37:40.337297 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-ac3f800c-3ff2-42f8-bbd9-c9cfa76d1c9b None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:40.338324 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-1c8d6172-ed0f-470d-99b9-47b62aba3f6d None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:40.338643 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ac3f800c-3ff2-42f8-bbd9-c9cfa76d1c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:40.339981 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-1c8d6172-ed0f-470d-99b9-47b62aba3f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:40.398240 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 777/1562] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:37:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:37:40.399526 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ad789-546c-4430-ab55-082d2dca7ad2 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:40.399930 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ad789-546c-4430-ab55-082d2dca7ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:40.400214 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ad789-546c-4430-ab55-082d2dca7ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:40.402162 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 786/1563] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:37:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:37:40.402162 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d6172-ed0f-470d-99b9-47b62aba3f6d None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:40.415509 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d6172-ed0f-470d-99b9-47b62aba3f6d None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:40.420730 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ddf617-1d56-4247-9411-d27e7b753a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:40.421401 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ddf617-1d56-4247-9411-d27e7b753a91 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:40.421675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ddf617-1d56-4247-9411-d27e7b753a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:40.421948 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ddf617-1d56-4247-9411-d27e7b753a91 admin admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:40.422468 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-214ad789-546c-4430-ab55-082d2dca7ad2 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:40.422901 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ddf617-1d56-4247-9411-d27e7b753a91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:37:40.423210 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ddf617-1d56-4247-9411-d27e7b753a91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:40.423663 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-214ad789-546c-4430-ab55-082d2dca7ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:40.424833 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ddf617-1d56-4247-9411-d27e7b753a91 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:40.432460 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 778/1564] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:37:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:37:40.437195 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ad789-546c-4430-ab55-082d2dca7ad2 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:40.437896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 787/1565] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:37:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:37:40.438706 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ddf617-1d56-4247-9411-d27e7b753a91 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:40.449680 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ad789-546c-4430-ab55-082d2dca7ad2 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:40.450001 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ddf617-1d56-4247-9411-d27e7b753a91 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:40.454343 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414d11a-5676-45b6-b371-aa41368a4c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:40.454770 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e1061d-ac0e-4022-b8f3-aad4f33d0ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:40.455004 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414d11a-5676-45b6-b371-aa41368a4c76 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:40.455298 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414d11a-5676-45b6-b371-aa41368a4c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:40.455562 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414d11a-5676-45b6-b371-aa41368a4c76 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:40.455676 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e1061d-ac0e-4022-b8f3-aad4f33d0ac0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:40.455903 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e1061d-ac0e-4022-b8f3-aad4f33d0ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:40.456178 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e1061d-ac0e-4022-b8f3-aad4f33d0ac0 admin admin] PATH_INFO: `/v3/projects/71e0c71509f94f158063991e37617894/users/b2103a7ed6824444a60fa5573e7ad39e/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:40.456296 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3414d11a-5676-45b6-b371-aa41368a4c76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:40.458179 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3414d11a-5676-45b6-b371-aa41368a4c76 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:40.473469 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e1061d-ac0e-4022-b8f3-aad4f33d0ac0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71e0c71509f94f158063991e37617894, user_id=b2103a7ed6824444a60fa5573e7ad39e, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:40.474529 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: WARNING py.warnings [None req-93e1061d-ac0e-4022-b8f3-aad4f33d0ac0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 19:37:40.474529 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: warnings.warn(msg) Jun 15 19:37:40.474529 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: Jun 15 19:37:40.476622 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e1061d-ac0e-4022-b8f3-aad4f33d0ac0 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:40.479160 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 779/1566] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:37:40 2022] POST /identity/v3/users => generated 290 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:37:40.480497 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414d11a-5676-45b6-b371-aa41368a4c76 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:40.491078 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414d11a-5676-45b6-b371-aa41368a4c76 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:40.491783 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 788/1567] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:37:40 2022] PUT /identity/v3/projects/71e0c71509f94f158063991e37617894/users/b2103a7ed6824444a60fa5573e7ad39e/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:37:40.492635 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e1061d-ac0e-4022-b8f3-aad4f33d0ac0 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:40.554749 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18437c36-92c6-40d9-9123-3a7a474f61df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:40.555437 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18437c36-92c6-40d9-9123-3a7a474f61df admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:40.555726 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18437c36-92c6-40d9-9123-3a7a474f61df admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:40.556012 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18437c36-92c6-40d9-9123-3a7a474f61df admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:40.556710 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18437c36-92c6-40d9-9123-3a7a474f61df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:40.558702 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18437c36-92c6-40d9-9123-3a7a474f61df admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:40.559699 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e1061d-ac0e-4022-b8f3-aad4f33d0ac0 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:40.564625 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f9db5-7c5b-4e8d-97de-1a95e781278f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:40.565390 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f9db5-7c5b-4e8d-97de-1a95e781278f admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:40.565676 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f9db5-7c5b-4e8d-97de-1a95e781278f admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:40.565950 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f9db5-7c5b-4e8d-97de-1a95e781278f admin admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:40.566714 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4f9db5-7c5b-4e8d-97de-1a95e781278f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:37:40.567017 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4f9db5-7c5b-4e8d-97de-1a95e781278f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:40.569189 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4f9db5-7c5b-4e8d-97de-1a95e781278f admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:40.578423 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 780/1568] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:37:40 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:37:40.579423 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18437c36-92c6-40d9-9123-3a7a474f61df admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:40.582414 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 789/1569] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:37:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:37:40.583273 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f9db5-7c5b-4e8d-97de-1a95e781278f admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:40.591136 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18437c36-92c6-40d9-9123-3a7a474f61df admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:40.593654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f9db5-7c5b-4e8d-97de-1a95e781278f admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:40.597171 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5185fa2-0269-4ae9-a435-ac7935f9a9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:40.597956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5185fa2-0269-4ae9-a435-ac7935f9a9bd admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:40.598281 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5185fa2-0269-4ae9-a435-ac7935f9a9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:40.598571 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5185fa2-0269-4ae9-a435-ac7935f9a9bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:40.598704 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ada736-dc81-44a1-b137-d07811cf9308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:40.599297 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5185fa2-0269-4ae9-a435-ac7935f9a9bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:37:40.599417 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ada736-dc81-44a1-b137-d07811cf9308 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:40.599588 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ada736-dc81-44a1-b137-d07811cf9308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:40.599706 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5185fa2-0269-4ae9-a435-ac7935f9a9bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:40.599867 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ada736-dc81-44a1-b137-d07811cf9308 admin admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:40.600604 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ada736-dc81-44a1-b137-d07811cf9308 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:37:40.600915 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ada736-dc81-44a1-b137-d07811cf9308 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:40.601201 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5185fa2-0269-4ae9-a435-ac7935f9a9bd admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:40.602550 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ada736-dc81-44a1-b137-d07811cf9308 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:40.613458 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 781/1570] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:37:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:37:40.614350 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5185fa2-0269-4ae9-a435-ac7935f9a9bd admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:40.616350 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 790/1571] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:37:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:37:40.617333 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ada736-dc81-44a1-b137-d07811cf9308 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:40.623851 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5185fa2-0269-4ae9-a435-ac7935f9a9bd admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:40.628779 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc0b2f-f120-46ae-b8c6-600bc8bfd422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:40.629425 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc0b2f-f120-46ae-b8c6-600bc8bfd422 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:40.629712 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc0b2f-f120-46ae-b8c6-600bc8bfd422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:40.629989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc0b2f-f120-46ae-b8c6-600bc8bfd422 admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:40.630774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcc0b2f-f120-46ae-b8c6-600bc8bfd422 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:37:40.631082 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcc0b2f-f120-46ae-b8c6-600bc8bfd422 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:40.632641 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ada736-dc81-44a1-b137-d07811cf9308 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:40.632752 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcc0b2f-f120-46ae-b8c6-600bc8bfd422 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:40.637586 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae4e060-4904-4f3e-a323-8175bcb5d3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:40.638248 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4e060-4904-4f3e-a323-8175bcb5d3c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:40.638534 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4e060-4904-4f3e-a323-8175bcb5d3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:40.638809 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4e060-4904-4f3e-a323-8175bcb5d3c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:40.640090 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae4e060-4904-4f3e-a323-8175bcb5d3c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:40.641944 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae4e060-4904-4f3e-a323-8175bcb5d3c0 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:40.644449 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 782/1572] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:37:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:37:40.645229 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc0b2f-f120-46ae-b8c6-600bc8bfd422 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:40.654549 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc0b2f-f120-46ae-b8c6-600bc8bfd422 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:40.659613 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325685b6-90e3-4f5e-9aa3-c5caf29a3e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:40.660247 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325685b6-90e3-4f5e-9aa3-c5caf29a3e43 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:40.660534 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325685b6-90e3-4f5e-9aa3-c5caf29a3e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:40.660814 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325685b6-90e3-4f5e-9aa3-c5caf29a3e43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:40.661531 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325685b6-90e3-4f5e-9aa3-c5caf29a3e43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:40.663151 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325685b6-90e3-4f5e-9aa3-c5caf29a3e43 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:40.674062 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 791/1573] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:37:40 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:37:40.674662 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 783/1574] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:37:40.675609 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325685b6-90e3-4f5e-9aa3-c5caf29a3e43 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:40.678381 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec14cc-45c8-4657-b100-3c6540917721 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:40.678670 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec14cc-45c8-4657-b100-3c6540917721 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:40.678946 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec14cc-45c8-4657-b100-3c6540917721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:40.685557 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325685b6-90e3-4f5e-9aa3-c5caf29a3e43 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:40.695739 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-9aec14cc-45c8-4657-b100-3c6540917721 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:40.696936 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-9aec14cc-45c8-4657-b100-3c6540917721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:40.702456 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:37:40.722552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:37:40.744684 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a336a9c-f6a9-48d1-ba98-e1909afe0220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:40.745346 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a336a9c-f6a9-48d1-ba98-e1909afe0220 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:40.745625 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a336a9c-f6a9-48d1-ba98-e1909afe0220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:40.745902 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a336a9c-f6a9-48d1-ba98-e1909afe0220 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:40.746714 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a336a9c-f6a9-48d1-ba98-e1909afe0220 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:40.748309 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a336a9c-f6a9-48d1-ba98-e1909afe0220 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:40.758275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 792/1575] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:37:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:37:40.758784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 784/1576] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:37:40.759223 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aec14cc-45c8-4657-b100-3c6540917721 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:40.759632 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a336a9c-f6a9-48d1-ba98-e1909afe0220 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:40.769500 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aec14cc-45c8-4657-b100-3c6540917721 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:40.770347 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a336a9c-f6a9-48d1-ba98-e1909afe0220 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:40.774070 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ab53a5-1d09-4d8e-970b-5471a0be570a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:40.774808 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ab53a5-1d09-4d8e-970b-5471a0be570a admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:40.775085 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ab53a5-1d09-4d8e-970b-5471a0be570a admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:40.775234 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929857dc-f116-4681-8312-96a0a0f3e5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:40.775357 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ab53a5-1d09-4d8e-970b-5471a0be570a admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:40.775912 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929857dc-f116-4681-8312-96a0a0f3e5d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:40.776022 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ab53a5-1d09-4d8e-970b-5471a0be570a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:40.776194 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929857dc-f116-4681-8312-96a0a0f3e5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:40.776470 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929857dc-f116-4681-8312-96a0a0f3e5d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:40.777284 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929857dc-f116-4681-8312-96a0a0f3e5d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:40.778037 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ab53a5-1d09-4d8e-970b-5471a0be570a admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:40.779215 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929857dc-f116-4681-8312-96a0a0f3e5d1 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:40.804921 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 793/1577] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:37:40 2022] POST /identity/v3/projects => generated 393 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:37:40.805715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ab53a5-1d09-4d8e-970b-5471a0be570a admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:40.806233 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 785/1578] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:37:40 2022] POST /identity/v3/projects => generated 405 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:37:40.806969 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929857dc-f116-4681-8312-96a0a0f3e5d1 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:40.870874 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929857dc-f116-4681-8312-96a0a0f3e5d1 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:40.871866 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ab53a5-1d09-4d8e-970b-5471a0be570a admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:40.875979 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad5d21-e1ac-4228-9fa4-a7a26fd24d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:40.876893 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad5d21-e1ac-4228-9fa4-a7a26fd24d51 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:40.877182 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad5d21-e1ac-4228-9fa4-a7a26fd24d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:40.877293 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dede65e-e1ab-47d2-8c28-a7d75961fa44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:40.877459 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad5d21-e1ac-4228-9fa4-a7a26fd24d51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:40.877857 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dede65e-e1ab-47d2-8c28-a7d75961fa44 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:40.878163 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dede65e-e1ab-47d2-8c28-a7d75961fa44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:40.878275 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dad5d21-e1ac-4228-9fa4-a7a26fd24d51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:40.878439 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dede65e-e1ab-47d2-8c28-a7d75961fa44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:40.879118 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dede65e-e1ab-47d2-8c28-a7d75961fa44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:40.880105 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dad5d21-e1ac-4228-9fa4-a7a26fd24d51 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:40.880983 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dede65e-e1ab-47d2-8c28-a7d75961fa44 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:40.907635 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 794/1579] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:37:40 2022] POST /identity/v3/projects => generated 357 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:37:40.908445 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dede65e-e1ab-47d2-8c28-a7d75961fa44 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:40.911024 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 786/1580] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:37:40 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:37:40.911870 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad5d21-e1ac-4228-9fa4-a7a26fd24d51 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:40.972663 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dede65e-e1ab-47d2-8c28-a7d75961fa44 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:40.973021 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad5d21-e1ac-4228-9fa4-a7a26fd24d51 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:40.977222 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf54f3a-a07b-4e8c-b5d0-839d10ffd1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:40.977850 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c31c2-fc8e-492c-b61a-a05f409b959b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:40.978011 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf54f3a-a07b-4e8c-b5d0-839d10ffd1ac admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:40.978185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf54f3a-a07b-4e8c-b5d0-839d10ffd1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:40.978460 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf54f3a-a07b-4e8c-b5d0-839d10ffd1ac admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:40.978645 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c31c2-fc8e-492c-b61a-a05f409b959b admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:40.978930 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c31c2-fc8e-492c-b61a-a05f409b959b admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:40.979201 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf54f3a-a07b-4e8c-b5d0-839d10ffd1ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:40.979306 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c31c2-fc8e-492c-b61a-a05f409b959b admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:40.980092 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793c31c2-fc8e-492c-b61a-a05f409b959b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:40.981099 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf54f3a-a07b-4e8c-b5d0-839d10ffd1ac admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:40.981727 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793c31c2-fc8e-492c-b61a-a05f409b959b admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:40.991883 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 787/1581] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:37:40.992644 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c31c2-fc8e-492c-b61a-a05f409b959b admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:41.001858 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 795/1582] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:37:40 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:37:41.002762 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf54f3a-a07b-4e8c-b5d0-839d10ffd1ac admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:41.002893 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c31c2-fc8e-492c-b61a-a05f409b959b admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:41.007623 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529de0c-0bee-4f60-a5fe-ca1e2ce5caf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:41.008314 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529de0c-0bee-4f60-a5fe-ca1e2ce5caf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:41.008597 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529de0c-0bee-4f60-a5fe-ca1e2ce5caf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:41.008873 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529de0c-0bee-4f60-a5fe-ca1e2ce5caf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:41.009625 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d529de0c-0bee-4f60-a5fe-ca1e2ce5caf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:41.011341 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d529de0c-0bee-4f60-a5fe-ca1e2ce5caf8 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:41.022875 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 788/1583] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:37:41.023742 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529de0c-0bee-4f60-a5fe-ca1e2ce5caf8 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:41.027076 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:37:41.031972 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:37:41.036466 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529de0c-0bee-4f60-a5fe-ca1e2ce5caf8 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:41.042067 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b947d2-b533-48ba-af4e-cb8fbe0af911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:41.042751 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b947d2-b533-48ba-af4e-cb8fbe0af911 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:41.043040 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b947d2-b533-48ba-af4e-cb8fbe0af911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:41.043322 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b947d2-b533-48ba-af4e-cb8fbe0af911 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:41.044004 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b947d2-b533-48ba-af4e-cb8fbe0af911 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:41.045996 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b947d2-b533-48ba-af4e-cb8fbe0af911 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:41.058743 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf54f3a-a07b-4e8c-b5d0-839d10ffd1ac admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:41.063358 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eafc6f-1c95-429f-8f4e-ac60068b60ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:41.064032 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eafc6f-1c95-429f-8f4e-ac60068b60ad service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:41.064327 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eafc6f-1c95-429f-8f4e-ac60068b60ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:41.064605 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eafc6f-1c95-429f-8f4e-ac60068b60ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:41.067814 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 789/1584] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:37:41 2022] POST /identity/v3/users => generated 297 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:37:41.068443 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b947d2-b533-48ba-af4e-cb8fbe0af911 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:41.077093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eafc6f-1c95-429f-8f4e-ac60068b60ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:41.078658 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eafc6f-1c95-429f-8f4e-ac60068b60ad service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:41.079796 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b947d2-b533-48ba-af4e-cb8fbe0af911 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:41.084347 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426ac53-3088-4f42-b400-b0f2e162af9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:41.085009 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426ac53-3088-4f42-b400-b0f2e162af9c admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:41.085318 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426ac53-3088-4f42-b400-b0f2e162af9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:41.085562 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426ac53-3088-4f42-b400-b0f2e162af9c admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:41.086262 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0426ac53-3088-4f42-b400-b0f2e162af9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:41.088138 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0426ac53-3088-4f42-b400-b0f2e162af9c admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:41.088512 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 796/1585] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:37:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:37:41.089280 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eafc6f-1c95-429f-8f4e-ac60068b60ad service neutron] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:41.099390 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eafc6f-1c95-429f-8f4e-ac60068b60ad service neutron] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:41.104648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e06760e-78fc-47f5-b6d6-b329d02de912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:41.105363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06760e-78fc-47f5-b6d6-b329d02de912 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:41.105631 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06760e-78fc-47f5-b6d6-b329d02de912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:41.105956 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06760e-78fc-47f5-b6d6-b329d02de912 admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:41.106986 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e06760e-78fc-47f5-b6d6-b329d02de912 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:41.109391 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e06760e-78fc-47f5-b6d6-b329d02de912 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:41.111429 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 790/1586] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:37:41 2022] POST /identity/v3/users => generated 303 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:37:41.114911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426ac53-3088-4f42-b400-b0f2e162af9c admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:41.125230 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426ac53-3088-4f42-b400-b0f2e162af9c admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:41.129918 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe6c922-7279-421a-bd1b-948ba9052cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:41.130077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 797/1587] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:37:41 2022] POST /identity/v3/users => generated 279 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:37:41.130623 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6c922-7279-421a-bd1b-948ba9052cd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:41.130965 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6c922-7279-421a-bd1b-948ba9052cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:41.131241 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6c922-7279-421a-bd1b-948ba9052cd2 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:41.131938 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe6c922-7279-421a-bd1b-948ba9052cd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:41.134318 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe6c922-7279-421a-bd1b-948ba9052cd2 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:41.134436 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e06760e-78fc-47f5-b6d6-b329d02de912 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:41.144735 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e06760e-78fc-47f5-b6d6-b329d02de912 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:41.149977 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a96f5-936e-4d85-8568-953aedd4bf09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:41.150904 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a96f5-936e-4d85-8568-953aedd4bf09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:41.151192 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a96f5-936e-4d85-8568-953aedd4bf09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:41.151550 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a96f5-936e-4d85-8568-953aedd4bf09 admin admin] PATH_INFO: `/v3/projects/240ef47b993b48dc9f48932a3c077a9a/users/c792103be95040719c1772cd8aa1d12f/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:41.156639 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 791/1588] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:37:41 2022] POST /identity/v3/users => generated 291 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:37:41.157443 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe6c922-7279-421a-bd1b-948ba9052cd2 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:41.168235 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a96f5-936e-4d85-8568-953aedd4bf09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=240ef47b993b48dc9f48932a3c077a9a, user_id=c792103be95040719c1772cd8aa1d12f, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:41.168435 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe6c922-7279-421a-bd1b-948ba9052cd2 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:41.172021 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a96f5-936e-4d85-8568-953aedd4bf09 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:41.174924 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c87be7-cdd1-4eb6-a539-926140dc3b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:41.175893 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c87be7-cdd1-4eb6-a539-926140dc3b6b admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:41.176232 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c87be7-cdd1-4eb6-a539-926140dc3b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:41.176587 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c87be7-cdd1-4eb6-a539-926140dc3b6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:41.177314 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c87be7-cdd1-4eb6-a539-926140dc3b6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:41.179014 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c87be7-cdd1-4eb6-a539-926140dc3b6b admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:41.186647 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 798/1589] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:37:41 2022] PUT /identity/v3/projects/240ef47b993b48dc9f48932a3c077a9a/users/c792103be95040719c1772cd8aa1d12f/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:37:41.188591 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a96f5-936e-4d85-8568-953aedd4bf09 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:41.197609 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 792/1590] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:37:41.197609 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c87be7-cdd1-4eb6-a539-926140dc3b6b admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:41.207692 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c87be7-cdd1-4eb6-a539-926140dc3b6b admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:41.261654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a96f5-936e-4d85-8568-953aedd4bf09 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:41.268261 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01dc261-162f-43c1-8006-8d5daf8fdee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:41.269154 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01dc261-162f-43c1-8006-8d5daf8fdee7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:41.269439 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01dc261-162f-43c1-8006-8d5daf8fdee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:41.269712 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01dc261-162f-43c1-8006-8d5daf8fdee7 admin admin] PATH_INFO: `/v3/projects/bd64a4f6222542f19ba5729f7abe12b2/users/c815423179f44b219e4f8572b10355c0/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:41.270019 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d3b0b-e4e3-49f3-9b30-82d0e3863dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:41.270732 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d3b0b-e4e3-49f3-9b30-82d0e3863dfe admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:41.271021 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d3b0b-e4e3-49f3-9b30-82d0e3863dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:41.271301 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d3b0b-e4e3-49f3-9b30-82d0e3863dfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:41.272039 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914d3b0b-e4e3-49f3-9b30-82d0e3863dfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:41.273685 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914d3b0b-e4e3-49f3-9b30-82d0e3863dfe admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:41.291557 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 793/1591] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:37:41.291557 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d3b0b-e4e3-49f3-9b30-82d0e3863dfe admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:41.300972 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d3b0b-e4e3-49f3-9b30-82d0e3863dfe admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:41.306436 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e4a02c-4e45-454b-ba61-4b76947254d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:41.307173 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01dc261-162f-43c1-8006-8d5daf8fdee7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd64a4f6222542f19ba5729f7abe12b2, user_id=c815423179f44b219e4f8572b10355c0, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:41.307326 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4a02c-4e45-454b-ba61-4b76947254d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:41.307763 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4a02c-4e45-454b-ba61-4b76947254d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:41.307997 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4a02c-4e45-454b-ba61-4b76947254d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:41.308805 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e4a02c-4e45-454b-ba61-4b76947254d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:41.310138 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01dc261-162f-43c1-8006-8d5daf8fdee7 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:41.310587 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e4a02c-4e45-454b-ba61-4b76947254d3 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:41.322176 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 794/1592] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:37:41.322991 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e4a02c-4e45-454b-ba61-4b76947254d3 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:41.326183 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 799/1593] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:37:41 2022] PUT /identity/v3/projects/bd64a4f6222542f19ba5729f7abe12b2/users/c815423179f44b219e4f8572b10355c0/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:37:41.330508 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01dc261-162f-43c1-8006-8d5daf8fdee7 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:41.336559 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e4a02c-4e45-454b-ba61-4b76947254d3 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:41.401669 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89da853-e1de-458b-bd6c-f14679fa66a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:41.402408 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89da853-e1de-458b-bd6c-f14679fa66a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:41.402844 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89da853-e1de-458b-bd6c-f14679fa66a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:41.403124 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89da853-e1de-458b-bd6c-f14679fa66a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:41.404052 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89da853-e1de-458b-bd6c-f14679fa66a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:41.405164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01dc261-162f-43c1-8006-8d5daf8fdee7 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:41.405528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89da853-e1de-458b-bd6c-f14679fa66a3 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:41.410271 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd8ec04-5846-4d90-8953-8f4eb911e163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:41.411048 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8ec04-5846-4d90-8953-8f4eb911e163 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:41.411335 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8ec04-5846-4d90-8953-8f4eb911e163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:41.411725 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8ec04-5846-4d90-8953-8f4eb911e163 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:41.412539 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd8ec04-5846-4d90-8953-8f4eb911e163 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:41.414292 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd8ec04-5846-4d90-8953-8f4eb911e163 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:41.416954 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 795/1594] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:37:41.418392 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa870b71-ed9a-44ba-9094-22990526659a None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:41.418820 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa870b71-ed9a-44ba-9094-22990526659a None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:41.419246 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa870b71-ed9a-44ba-9094-22990526659a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:41.426508 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 800/1595] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:37:41.427409 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd8ec04-5846-4d90-8953-8f4eb911e163 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:41.435417 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:37:41.440547 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd8ec04-5846-4d90-8953-8f4eb911e163 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:41.445068 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-aa870b71-ed9a-44ba-9094-22990526659a None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:41.446704 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0c126-8fe6-4c23-a29c-3be3939948cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:41.446875 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-aa870b71-ed9a-44ba-9094-22990526659a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:41.448028 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0c126-8fe6-4c23-a29c-3be3939948cd admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:41.448537 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0c126-8fe6-4c23-a29c-3be3939948cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:41.449021 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0c126-8fe6-4c23-a29c-3be3939948cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:41.449973 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b0c126-8fe6-4c23-a29c-3be3939948cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:41.451743 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b0c126-8fe6-4c23-a29c-3be3939948cd admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:41.456649 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 796/1596] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:37:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:37:41.458038 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa870b71-ed9a-44ba-9094-22990526659a None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:41.465676 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 801/1597] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:37:41.469276 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0c126-8fe6-4c23-a29c-3be3939948cd admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:41.477489 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0c126-8fe6-4c23-a29c-3be3939948cd admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:41.481291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa870b71-ed9a-44ba-9094-22990526659a None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:41.488618 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a931413-3169-4504-9c91-afe3e4308bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:41.488618 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a931413-3169-4504-9c91-afe3e4308bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:41.491789 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a931413-3169-4504-9c91-afe3e4308bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:41.491789 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a931413-3169-4504-9c91-afe3e4308bd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:41.491789 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a931413-3169-4504-9c91-afe3e4308bd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:41.493487 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be1b24-dfae-4c62-9995-78589a2b1e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:41.494423 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be1b24-dfae-4c62-9995-78589a2b1e1c admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:41.494547 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a931413-3169-4504-9c91-afe3e4308bd8 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:41.494961 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be1b24-dfae-4c62-9995-78589a2b1e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:41.495464 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be1b24-dfae-4c62-9995-78589a2b1e1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:41.496413 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be1b24-dfae-4c62-9995-78589a2b1e1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:41.498350 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be1b24-dfae-4c62-9995-78589a2b1e1c admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:41.509893 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 797/1598] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:37:41.510185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 802/1599] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:37:41.511049 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97c50a-f76b-4d72-938c-b5c223a8e84b None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:41.511171 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a931413-3169-4504-9c91-afe3e4308bd8 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:41.511332 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97c50a-f76b-4d72-938c-b5c223a8e84b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:41.511610 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97c50a-f76b-4d72-938c-b5c223a8e84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:41.521085 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:37:41.522886 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a931413-3169-4504-9c91-afe3e4308bd8 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:41.528659 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee6b451-110d-46e8-bbed-cbf9d9ddca9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:41.529339 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee6b451-110d-46e8-bbed-cbf9d9ddca9c admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:41.529683 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee6b451-110d-46e8-bbed-cbf9d9ddca9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:41.529994 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee6b451-110d-46e8-bbed-cbf9d9ddca9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:41.531075 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee6b451-110d-46e8-bbed-cbf9d9ddca9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:41.532866 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee6b451-110d-46e8-bbed-cbf9d9ddca9c admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:41.535723 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-9e97c50a-f76b-4d72-938c-b5c223a8e84b None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:41.537135 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-9e97c50a-f76b-4d72-938c-b5c223a8e84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:41.546288 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 798/1600] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:37:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:37:41.546968 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 803/1601] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:37:41.547288 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e97c50a-f76b-4d72-938c-b5c223a8e84b None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:41.548397 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee6b451-110d-46e8-bbed-cbf9d9ddca9c admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:41.559553 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee6b451-110d-46e8-bbed-cbf9d9ddca9c admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:41.560190 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e97c50a-f76b-4d72-938c-b5c223a8e84b None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:41.566425 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c633677-a03d-40ce-80ed-ad5e1f264dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:41.567057 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4803e-3508-440f-a211-eb1058d104f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:41.567812 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4803e-3508-440f-a211-eb1058d104f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:41.568211 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4803e-3508-440f-a211-eb1058d104f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:41.568334 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c633677-a03d-40ce-80ed-ad5e1f264dc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:41.568496 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4803e-3508-440f-a211-eb1058d104f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:41.568964 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c633677-a03d-40ce-80ed-ad5e1f264dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:41.569439 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4803e-3508-440f-a211-eb1058d104f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:41.569552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c633677-a03d-40ce-80ed-ad5e1f264dc3 admin admin] PATH_INFO: `/v3/projects/e8e1414ffa3548938432aed200c75736/users/e1635576dfe242f88ec1abb7be6359fb/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:41.571246 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4803e-3508-440f-a211-eb1058d104f8 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:41.585349 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 799/1602] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:37:41.585349 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4803e-3508-440f-a211-eb1058d104f8 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:41.585349 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7fd4803e-3508-440f-a211-eb1058d104f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:41.591582 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c633677-a03d-40ce-80ed-ad5e1f264dc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8e1414ffa3548938432aed200c75736, user_id=e1635576dfe242f88ec1abb7be6359fb, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:41.595206 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c633677-a03d-40ce-80ed-ad5e1f264dc3 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:41.612316 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 804/1603] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:37:41 2022] PUT /identity/v3/projects/e8e1414ffa3548938432aed200c75736/users/e1635576dfe242f88ec1abb7be6359fb/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:37:41.614269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c633677-a03d-40ce-80ed-ad5e1f264dc3 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:41.662107 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4803e-3508-440f-a211-eb1058d104f8 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:41.682471 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c633677-a03d-40ce-80ed-ad5e1f264dc3 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:41.687161 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c92d644-6d97-4fad-842e-039c922adf40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:41.688059 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92d644-6d97-4fad-842e-039c922adf40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:41.688362 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92d644-6d97-4fad-842e-039c922adf40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:41.688631 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92d644-6d97-4fad-842e-039c922adf40 admin admin] PATH_INFO: `/v3/projects/0d5270cd64fd4929af7b063cf2e2b653/users/f26a832b69564f2a8387215afba6730e/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:41.706009 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c92d644-6d97-4fad-842e-039c922adf40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d5270cd64fd4929af7b063cf2e2b653, user_id=f26a832b69564f2a8387215afba6730e, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:41.708766 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c92d644-6d97-4fad-842e-039c922adf40 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:41.722064 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c719eec-06ea-4184-b9b7-beeabb2d312f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:41.722912 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c719eec-06ea-4184-b9b7-beeabb2d312f service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:41.723093 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c719eec-06ea-4184-b9b7-beeabb2d312f service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:41.723374 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c719eec-06ea-4184-b9b7-beeabb2d312f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:41.730337 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 805/1604] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:37:41 2022] PUT /identity/v3/projects/0d5270cd64fd4929af7b063cf2e2b653/users/f26a832b69564f2a8387215afba6730e/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:37:41.731420 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c92d644-6d97-4fad-842e-039c922adf40 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:41.733430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c719eec-06ea-4184-b9b7-beeabb2d312f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:41.734924 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c719eec-06ea-4184-b9b7-beeabb2d312f service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:41.745590 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 800/1605] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:37:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:37:41.746411 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c719eec-06ea-4184-b9b7-beeabb2d312f service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:41.757524 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c719eec-06ea-4184-b9b7-beeabb2d312f service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:41.810984 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c92d644-6d97-4fad-842e-039c922adf40 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:41.816513 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a5180-5a5f-4eb1-ac48-9dc3e1328935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:41.817409 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a5180-5a5f-4eb1-ac48-9dc3e1328935 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:41.817696 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a5180-5a5f-4eb1-ac48-9dc3e1328935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:41.817999 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a5180-5a5f-4eb1-ac48-9dc3e1328935 admin admin] PATH_INFO: `/v3/projects/b392b3914e7f4a1b93bbfc0f3ab8cfc1/users/97c5493f93ea4e51ac0491a65701845c/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:41.830825 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62236987-1e2b-4740-9b93-172821d0c0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:41.832818 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62236987-1e2b-4740-9b93-172821d0c0d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:41.833227 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62236987-1e2b-4740-9b93-172821d0c0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:41.833513 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62236987-1e2b-4740-9b93-172821d0c0d2 admin admin] PATH_INFO: `/v3/projects/6f098b8fc390430b9f98879e6afcd1b7/users/9c6fd3dfbd5a485c9b3c6091bf18667d/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:41.837225 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7a5180-5a5f-4eb1-ac48-9dc3e1328935 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b392b3914e7f4a1b93bbfc0f3ab8cfc1, user_id=97c5493f93ea4e51ac0491a65701845c, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:41.840131 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7a5180-5a5f-4eb1-ac48-9dc3e1328935 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:41.853626 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62236987-1e2b-4740-9b93-172821d0c0d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f098b8fc390430b9f98879e6afcd1b7, user_id=9c6fd3dfbd5a485c9b3c6091bf18667d, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:41.856451 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 806/1606] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:37:41 2022] PUT /identity/v3/projects/b392b3914e7f4a1b93bbfc0f3ab8cfc1/users/97c5493f93ea4e51ac0491a65701845c/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:37:41.857067 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a5180-5a5f-4eb1-ac48-9dc3e1328935 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:41.861082 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62236987-1e2b-4740-9b93-172821d0c0d2 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:41.873819 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:37:41.889068 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 801/1607] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:37:41 2022] PUT /identity/v3/projects/6f098b8fc390430b9f98879e6afcd1b7/users/9c6fd3dfbd5a485c9b3c6091bf18667d/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:37:41.890077 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62236987-1e2b-4740-9b93-172821d0c0d2 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:41.940050 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a5180-5a5f-4eb1-ac48-9dc3e1328935 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:41.949718 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:37:42.016951 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62236987-1e2b-4740-9b93-172821d0c0d2 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:42.021938 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186cf006-8c78-45e4-96a1-69507d6f468d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:42.023164 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186cf006-8c78-45e4-96a1-69507d6f468d admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:42.023553 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186cf006-8c78-45e4-96a1-69507d6f468d admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:42.023950 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186cf006-8c78-45e4-96a1-69507d6f468d admin admin] PATH_INFO: `/v3/projects/03f70bdddb8d4e808b0eade72feaadda/users/1a23e828921f4344bcef652c6d4a5db6/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:42.041192 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186cf006-8c78-45e4-96a1-69507d6f468d admin admin] RBAC: Authorizing `identity:create_grant(project_id=03f70bdddb8d4e808b0eade72feaadda, user_id=1a23e828921f4344bcef652c6d4a5db6, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:42.044446 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186cf006-8c78-45e4-96a1-69507d6f468d admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:42.052718 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abecdec1-3a81-4834-8ef6-fa8652169d1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:42.054228 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecdec1-3a81-4834-8ef6-fa8652169d1b service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:42.054693 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecdec1-3a81-4834-8ef6-fa8652169d1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:42.055325 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecdec1-3a81-4834-8ef6-fa8652169d1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:42.063622 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 802/1608] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:37:41 2022] PUT /identity/v3/projects/03f70bdddb8d4e808b0eade72feaadda/users/1a23e828921f4344bcef652c6d4a5db6/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:37:42.070485 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abecdec1-3a81-4834-8ef6-fa8652169d1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:42.072897 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abecdec1-3a81-4834-8ef6-fa8652169d1b service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:42.075496 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f66b43-0495-4bb5-b161-e4a3408de2c7 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:42.075867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f66b43-0495-4bb5-b161-e4a3408de2c7 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:42.076234 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f66b43-0495-4bb5-b161-e4a3408de2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:42.096015 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 807/1609] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:37:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:37:42.098747 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-53f66b43-0495-4bb5-b161-e4a3408de2c7 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:42.100356 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-53f66b43-0495-4bb5-b161-e4a3408de2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:42.100866 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab42ad0-3d9e-4317-91cd-8abc793f49aa None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:42.102069 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab42ad0-3d9e-4317-91cd-8abc793f49aa None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:42.102853 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab42ad0-3d9e-4317-91cd-8abc793f49aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:42.139875 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-0ab42ad0-3d9e-4317-91cd-8abc793f49aa None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:42.141127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-0ab42ad0-3d9e-4317-91cd-8abc793f49aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:42.194142 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 803/1610] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:37:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:37:42.196640 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bbf130-f58d-4ff4-8e5f-832edce596e6 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:42.197046 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bbf130-f58d-4ff4-8e5f-832edce596e6 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:42.197457 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bbf130-f58d-4ff4-8e5f-832edce596e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:42.220874 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-84bbf130-f58d-4ff4-8e5f-832edce596e6 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:42.223028 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-84bbf130-f58d-4ff4-8e5f-832edce596e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:42.241088 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 804/1611] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:37:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:37:42.244065 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0201d2-c0d4-47ce-84f5-1132dbf29fd9 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:42.244365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0201d2-c0d4-47ce-84f5-1132dbf29fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:42.244648 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0201d2-c0d4-47ce-84f5-1132dbf29fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:42.258749 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 808/1612] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:37:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:37:42.259836 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e9bf2-e6b8-4b0d-833b-eedb720d3857 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:42.260129 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e9bf2-e6b8-4b0d-833b-eedb720d3857 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:42.260406 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e9bf2-e6b8-4b0d-833b-eedb720d3857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:42.288899 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-6b0201d2-c0d4-47ce-84f5-1132dbf29fd9 None None] MFA Rules not processed for user `9c6fd3dfbd5a485c9b3c6091bf18667d`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:42.290258 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6b0201d2-c0d4-47ce-84f5-1132dbf29fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:42.340224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-256e9bf2-e6b8-4b0d-833b-eedb720d3857 None None] MFA Rules not processed for user `1a23e828921f4344bcef652c6d4a5db6`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:42.343107 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-256e9bf2-e6b8-4b0d-833b-eedb720d3857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:42.393822 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 805/1613] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:37:42 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:37:42.393822 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0201d2-c0d4-47ce-84f5-1132dbf29fd9 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:42.450319 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:37:42.489367 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:37:42.493490 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0201d2-c0d4-47ce-84f5-1132dbf29fd9 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:42.499588 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 809/1614] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:37:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:37:42.504334 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e9bf2-e6b8-4b0d-833b-eedb720d3857 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:42.508446 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7409cc27-a154-401a-b908-14b340f6c06c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:42.509968 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409cc27-a154-401a-b908-14b340f6c06c service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:42.510655 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409cc27-a154-401a-b908-14b340f6c06c service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:42.510655 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409cc27-a154-401a-b908-14b340f6c06c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:42.524883 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e9bf2-e6b8-4b0d-833b-eedb720d3857 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:42.534545 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7409cc27-a154-401a-b908-14b340f6c06c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:42.535438 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d63bb1-0e04-4e78-8590-a2bda1cc0209 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:42.537775 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d63bb1-0e04-4e78-8590-a2bda1cc0209 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:42.538064 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d63bb1-0e04-4e78-8590-a2bda1cc0209 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:42.538626 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7409cc27-a154-401a-b908-14b340f6c06c service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:42.539623 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d63bb1-0e04-4e78-8590-a2bda1cc0209 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:42.563244 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d63bb1-0e04-4e78-8590-a2bda1cc0209 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:42.569867 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d63bb1-0e04-4e78-8590-a2bda1cc0209 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:42.574124 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 806/1615] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:37:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:37:42.577179 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7409cc27-a154-401a-b908-14b340f6c06c service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:42.595411 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 810/1616] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:37:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:37:42.595966 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7409cc27-a154-401a-b908-14b340f6c06c service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:42.600474 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa60b85-4beb-4ba4-9b2d-1d806edc2177 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:42.601626 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d63bb1-0e04-4e78-8590-a2bda1cc0209 service neutron] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:42.602213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa60b85-4beb-4ba4-9b2d-1d806edc2177 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:42.602673 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa60b85-4beb-4ba4-9b2d-1d806edc2177 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:42.602961 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa60b85-4beb-4ba4-9b2d-1d806edc2177 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:42.611917 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d63bb1-0e04-4e78-8590-a2bda1cc0209 service neutron] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:42.616936 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77c0a49-cb00-48a8-9f2e-857851b5eccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:42.617599 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c0a49-cb00-48a8-9f2e-857851b5eccc admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:42.617885 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c0a49-cb00-48a8-9f2e-857851b5eccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:42.618225 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c0a49-cb00-48a8-9f2e-857851b5eccc admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:42.618934 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77c0a49-cb00-48a8-9f2e-857851b5eccc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:42.620769 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa60b85-4beb-4ba4-9b2d-1d806edc2177 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:42.621957 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77c0a49-cb00-48a8-9f2e-857851b5eccc admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:42.624688 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa60b85-4beb-4ba4-9b2d-1d806edc2177 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:42.654375 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 807/1617] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:37:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:37:42.655092 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 811/1618] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:37:42 2022] POST /identity/v3/projects => generated 357 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:37:42.655298 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa60b85-4beb-4ba4-9b2d-1d806edc2177 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:42.659558 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77c0a49-cb00-48a8-9f2e-857851b5eccc admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:42.680991 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa60b85-4beb-4ba4-9b2d-1d806edc2177 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:42.742110 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77c0a49-cb00-48a8-9f2e-857851b5eccc admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:42.750439 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9200473-f0ba-4bb5-853f-899ea99f4585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:42.751776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9200473-f0ba-4bb5-853f-899ea99f4585 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:42.752058 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9200473-f0ba-4bb5-853f-899ea99f4585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:42.752564 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9200473-f0ba-4bb5-853f-899ea99f4585 admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:42.753642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9200473-f0ba-4bb5-853f-899ea99f4585 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:42.755785 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9200473-f0ba-4bb5-853f-899ea99f4585 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:42.781740 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3638df-cee3-431e-9d94-e2a979426a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:42.782492 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3638df-cee3-431e-9d94-e2a979426a15 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:42.782786 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3638df-cee3-431e-9d94-e2a979426a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:42.783067 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3638df-cee3-431e-9d94-e2a979426a15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:42.784000 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3638df-cee3-431e-9d94-e2a979426a15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:42.786975 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3638df-cee3-431e-9d94-e2a979426a15 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:42.789528 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 812/1619] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:37:42 2022] POST /identity/v3/users => generated 285 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:37:42.793533 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9200473-f0ba-4bb5-853f-899ea99f4585 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:42.814606 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9200473-f0ba-4bb5-853f-899ea99f4585 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:42.828724 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 808/1620] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:37:42 2022] POST /identity/v3/projects => generated 381 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:37:42.833636 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3638df-cee3-431e-9d94-e2a979426a15 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:42.837780 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ef573-eb1c-4d0b-9fdb-d09f3db846fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:42.838893 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ef573-eb1c-4d0b-9fdb-d09f3db846fd admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:42.839242 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ef573-eb1c-4d0b-9fdb-d09f3db846fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:42.839525 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ef573-eb1c-4d0b-9fdb-d09f3db846fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:42.840598 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8ef573-eb1c-4d0b-9fdb-d09f3db846fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:42.845539 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8ef573-eb1c-4d0b-9fdb-d09f3db846fd admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:42.873609 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 813/1621] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:37:42.878286 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ef573-eb1c-4d0b-9fdb-d09f3db846fd admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:42.912363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ef573-eb1c-4d0b-9fdb-d09f3db846fd admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:42.993762 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3638df-cee3-431e-9d94-e2a979426a15 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:43.011352 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b35fc1d-4632-42b8-8f20-9bbdcab3e00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:43.012219 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35fc1d-4632-42b8-8f20-9bbdcab3e00c admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:43.012513 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35fc1d-4632-42b8-8f20-9bbdcab3e00c admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:43.012794 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35fc1d-4632-42b8-8f20-9bbdcab3e00c admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:43.013507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b35fc1d-4632-42b8-8f20-9bbdcab3e00c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:43.018622 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b35fc1d-4632-42b8-8f20-9bbdcab3e00c admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:43.043885 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c62d58-8c26-43c7-ad39-18ecebcae016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:43.044982 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c62d58-8c26-43c7-ad39-18ecebcae016 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:43.045971 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c62d58-8c26-43c7-ad39-18ecebcae016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:43.046225 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c62d58-8c26-43c7-ad39-18ecebcae016 admin admin] PATH_INFO: `/v3/projects/db667a1089aa45f8bbc26f89497ab07a/users/ac80c0e9b3404d588100672e2b25ed55/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:43.062803 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 809/1622] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:37:42 2022] POST /identity/v3/users => generated 297 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:37:43.064009 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b35fc1d-4632-42b8-8f20-9bbdcab3e00c admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:43.086413 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b35fc1d-4632-42b8-8f20-9bbdcab3e00c admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:43.088204 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c62d58-8c26-43c7-ad39-18ecebcae016 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db667a1089aa45f8bbc26f89497ab07a, user_id=ac80c0e9b3404d588100672e2b25ed55, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:43.091227 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c62d58-8c26-43c7-ad39-18ecebcae016 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:43.100501 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ce012-a60d-44b8-9402-8939053c6b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:43.101263 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ce012-a60d-44b8-9402-8939053c6b82 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:43.101606 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ce012-a60d-44b8-9402-8939053c6b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:43.101931 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ce012-a60d-44b8-9402-8939053c6b82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:43.102949 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ce012-a60d-44b8-9402-8939053c6b82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:43.104769 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ce012-a60d-44b8-9402-8939053c6b82 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:43.119565 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 814/1623] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:37:42 2022] PUT /identity/v3/projects/db667a1089aa45f8bbc26f89497ab07a/users/ac80c0e9b3404d588100672e2b25ed55/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:37:43.123858 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c62d58-8c26-43c7-ad39-18ecebcae016 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:43.137714 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 810/1624] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:37:43.138916 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ce012-a60d-44b8-9402-8939053c6b82 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:43.163663 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ce012-a60d-44b8-9402-8939053c6b82 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:43.194041 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:37:43.228598 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:37:43.265972 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c62d58-8c26-43c7-ad39-18ecebcae016 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:43.281690 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394d120-ed69-433c-aa3e-776c7ef72841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:43.282204 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394d120-ed69-433c-aa3e-776c7ef72841 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:43.282648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394d120-ed69-433c-aa3e-776c7ef72841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:43.283028 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394d120-ed69-433c-aa3e-776c7ef72841 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:43.283843 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9394d120-ed69-433c-aa3e-776c7ef72841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:43.285735 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9394d120-ed69-433c-aa3e-776c7ef72841 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:43.298141 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 815/1625] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:37:43.309252 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394d120-ed69-433c-aa3e-776c7ef72841 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:43.312914 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f943dfc-4e64-4fd3-86c4-2b9de8173481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:43.322666 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f943dfc-4e64-4fd3-86c4-2b9de8173481 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:43.322666 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f943dfc-4e64-4fd3-86c4-2b9de8173481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:43.322666 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f943dfc-4e64-4fd3-86c4-2b9de8173481 admin admin] PATH_INFO: `/v3/projects/12496d3308984f88a079e8ce8596d03a/users/63d4354f0e63436abaac2e27942f0cab/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:43.325967 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394d120-ed69-433c-aa3e-776c7ef72841 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:43.338737 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a20bf4-ac48-4898-8a87-50749f8497da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:43.339649 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a20bf4-ac48-4898-8a87-50749f8497da admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:43.342986 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a20bf4-ac48-4898-8a87-50749f8497da admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:43.343461 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a20bf4-ac48-4898-8a87-50749f8497da admin admin] PATH_INFO: `/v3/projects/db667a1089aa45f8bbc26f89497ab07a/users/ac80c0e9b3404d588100672e2b25ed55/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:43.345693 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a20bf4-ac48-4898-8a87-50749f8497da admin admin] RBAC: Authorizing `identity:create_grant(project_id=db667a1089aa45f8bbc26f89497ab07a, user_id=ac80c0e9b3404d588100672e2b25ed55, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:43.348409 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a20bf4-ac48-4898-8a87-50749f8497da admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:43.367422 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 816/1626] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:37:43 2022] PUT /identity/v3/projects/db667a1089aa45f8bbc26f89497ab07a/users/ac80c0e9b3404d588100672e2b25ed55/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:37:43.369396 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f943dfc-4e64-4fd3-86c4-2b9de8173481 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12496d3308984f88a079e8ce8596d03a, user_id=63d4354f0e63436abaac2e27942f0cab, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:43.369741 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c119343-3b9c-4094-b15b-70189e270254 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:43.370006 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c119343-3b9c-4094-b15b-70189e270254 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:43.370359 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c119343-3b9c-4094-b15b-70189e270254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:43.372086 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f943dfc-4e64-4fd3-86c4-2b9de8173481 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:43.408215 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 811/1627] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:37:43 2022] PUT /identity/v3/projects/12496d3308984f88a079e8ce8596d03a/users/63d4354f0e63436abaac2e27942f0cab/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:37:43.409560 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ccfcc-704e-41b3-b91f-fdcb357fae9e None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:43.409851 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ccfcc-704e-41b3-b91f-fdcb357fae9e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:43.414446 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ccfcc-704e-41b3-b91f-fdcb357fae9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:43.424028 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-2c119343-3b9c-4094-b15b-70189e270254 None None] MFA Rules not processed for user `c792103be95040719c1772cd8aa1d12f`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:43.425664 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-2c119343-3b9c-4094-b15b-70189e270254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:43.474773 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-d81ccfcc-704e-41b3-b91f-fdcb357fae9e None None] MFA Rules not processed for user `ac80c0e9b3404d588100672e2b25ed55`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:43.478597 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d81ccfcc-704e-41b3-b91f-fdcb357fae9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:43.678553 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 817/1628] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:37:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:37:43.678553 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c119343-3b9c-4094-b15b-70189e270254 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:43.747837 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 812/1629] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:37:43 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:37:43.749889 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12d4d5-aa0d-4405-8f8b-5617ef2fd08d None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:43.750356 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12d4d5-aa0d-4405-8f8b-5617ef2fd08d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:43.750799 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12d4d5-aa0d-4405-8f8b-5617ef2fd08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:43.783220 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-ff12d4d5-aa0d-4405-8f8b-5617ef2fd08d None None] MFA Rules not processed for user `c792103be95040719c1772cd8aa1d12f`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:43.784730 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ff12d4d5-aa0d-4405-8f8b-5617ef2fd08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:43.808813 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 813/1630] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:37:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:37:43.809689 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12d4d5-aa0d-4405-8f8b-5617ef2fd08d None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:43.954453 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12d4d5-aa0d-4405-8f8b-5617ef2fd08d None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:43.965345 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd1ed3b-3f36-4efe-8968-68fad733c3a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:43.968470 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1ed3b-3f36-4efe-8968-68fad733c3a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:43.969214 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1ed3b-3f36-4efe-8968-68fad733c3a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:43.970171 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1ed3b-3f36-4efe-8968-68fad733c3a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:43.994343 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c119343-3b9c-4094-b15b-70189e270254 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:43.998748 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd1ed3b-3f36-4efe-8968-68fad733c3a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:44.001856 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd1ed3b-3f36-4efe-8968-68fad733c3a8 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:44.017040 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd074200-2899-41a8-bf77-5f564df3e91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:44.018218 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd074200-2899-41a8-bf77-5f564df3e91c admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:44.025646 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd074200-2899-41a8-bf77-5f564df3e91c admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:44.026138 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd074200-2899-41a8-bf77-5f564df3e91c admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:44.027587 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd074200-2899-41a8-bf77-5f564df3e91c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:44.041368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd074200-2899-41a8-bf77-5f564df3e91c admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:44.046327 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 814/1631] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:37:43 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:37:44.049157 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de7b2b-b2cd-4cfd-8fcc-aed5a4775956 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:44.049665 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de7b2b-b2cd-4cfd-8fcc-aed5a4775956 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:44.050315 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de7b2b-b2cd-4cfd-8fcc-aed5a4775956 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:44.059808 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 815/1632] 10.222.0.15 () {54 vars in 961 bytes} [Wed Jun 15 19:37:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:37:44.068340 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ca4df-1a63-4521-be36-055159c843d5 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:44.069273 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ca4df-1a63-4521-be36-055159c843d5 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:44.070370 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ca4df-1a63-4521-be36-055159c843d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:44.085033 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 818/1633] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 436 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:37:44.087187 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd074200-2899-41a8-bf77-5f564df3e91c admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:44.101629 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-259ca4df-1a63-4521-be36-055159c843d5 None None] MFA Rules not processed for user `3e74ac44e5814122a3226ac023f3688a`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:44.103326 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-259ca4df-1a63-4521-be36-055159c843d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:44.128413 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd074200-2899-41a8-bf77-5f564df3e91c admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:44.144207 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff7301-0580-4493-9ddb-c1a9aa344740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:44.145529 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff7301-0580-4493-9ddb-c1a9aa344740 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:44.146024 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff7301-0580-4493-9ddb-c1a9aa344740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:44.146423 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff7301-0580-4493-9ddb-c1a9aa344740 admin admin] PATH_INFO: `/v3/projects/12496d3308984f88a079e8ce8596d03a/users/63d4354f0e63436abaac2e27942f0cab/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:44.152236 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ff7301-0580-4493-9ddb-c1a9aa344740 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12496d3308984f88a079e8ce8596d03a, user_id=63d4354f0e63436abaac2e27942f0cab, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:44.155431 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ff7301-0580-4493-9ddb-c1a9aa344740 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:44.156145 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:37:44.198431 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 819/1634] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:37:44 2022] PUT /identity/v3/projects/12496d3308984f88a079e8ce8596d03a/users/63d4354f0e63436abaac2e27942f0cab/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:37:44.200318 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69144d75-276b-44f0-9dbe-85425a938892 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:44.200556 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69144d75-276b-44f0-9dbe-85425a938892 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:44.201120 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69144d75-276b-44f0-9dbe-85425a938892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:44.234899 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 816/1635] 10.222.0.15 () {58 vars in 1058 bytes} [Wed Jun 15 19:37:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:37:44.242843 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ca4df-1a63-4521-be36-055159c843d5 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:44.247206 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-69144d75-276b-44f0-9dbe-85425a938892 None None] MFA Rules not processed for user `63d4354f0e63436abaac2e27942f0cab`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:44.249190 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-69144d75-276b-44f0-9dbe-85425a938892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:44.256144 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ca4df-1a63-4521-be36-055159c843d5 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:44.264033 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:37:44.330773 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51921b-26c6-4b15-9db5-7df822ef1554 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:44.331945 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51921b-26c6-4b15-9db5-7df822ef1554 service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:44.332455 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51921b-26c6-4b15-9db5-7df822ef1554 service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:44.332973 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51921b-26c6-4b15-9db5-7df822ef1554 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:44.436579 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51921b-26c6-4b15-9db5-7df822ef1554 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:44.441162 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51921b-26c6-4b15-9db5-7df822ef1554 service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:44.444776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 820/1636] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:37:44 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:37:44.452296 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69144d75-276b-44f0-9dbe-85425a938892 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:44.468303 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 817/1637] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:37:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:37:44.469225 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51921b-26c6-4b15-9db5-7df822ef1554 service nova] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:44.563959 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51921b-26c6-4b15-9db5-7df822ef1554 service nova] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:44.572810 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855f473-a268-47ab-8ca3-b21ce82d81e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:44.573861 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855f473-a268-47ab-8ca3-b21ce82d81e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:44.573982 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855f473-a268-47ab-8ca3-b21ce82d81e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:44.574294 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855f473-a268-47ab-8ca3-b21ce82d81e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:44.599351 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69144d75-276b-44f0-9dbe-85425a938892 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:44.612260 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0855f473-a268-47ab-8ca3-b21ce82d81e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:44.614336 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0855f473-a268-47ab-8ca3-b21ce82d81e4 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:44.618101 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b1818-95da-4a39-b2a9-2fa92d78416f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:44.619984 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b1818-95da-4a39-b2a9-2fa92d78416f service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:44.621747 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b1818-95da-4a39-b2a9-2fa92d78416f service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:44.622185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b1818-95da-4a39-b2a9-2fa92d78416f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:44.641567 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 818/1638] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:37:44 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:37:44.762240 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0b1818-95da-4a39-b2a9-2fa92d78416f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:44.766245 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0b1818-95da-4a39-b2a9-2fa92d78416f service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:44.783250 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855f473-a268-47ab-8ca3-b21ce82d81e4 service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:44.791063 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:37:44.792328 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 821/1639] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:37:44 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:37:44.796425 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855f473-a268-47ab-8ca3-b21ce82d81e4 service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:44.802766 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c20b90b-fddc-4507-8733-ffda7b0c4778 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:44.803660 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c20b90b-fddc-4507-8733-ffda7b0c4778 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:44.804149 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c20b90b-fddc-4507-8733-ffda7b0c4778 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:44.804649 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c20b90b-fddc-4507-8733-ffda7b0c4778 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:44.933692 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c20b90b-fddc-4507-8733-ffda7b0c4778 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:44.938082 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c20b90b-fddc-4507-8733-ffda7b0c4778 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:44.959982 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 819/1640] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:37:44 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:37:44.991281 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d028f2-2e66-44f1-82c9-4355e9ec803c None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:44.991613 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d028f2-2e66-44f1-82c9-4355e9ec803c None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:44.991921 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d028f2-2e66-44f1-82c9-4355e9ec803c None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:44.993471 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 822/1641] 10.222.0.15 () {54 vars in 912 bytes} [Wed Jun 15 19:37:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:37:45.001466 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b073c-e588-405a-9510-ef7c7fde13d1 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:45.002254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b073c-e588-405a-9510-ef7c7fde13d1 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:45.002884 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b073c-e588-405a-9510-ef7c7fde13d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:45.027941 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-ea3b073c-e588-405a-9510-ef7c7fde13d1 None None] MFA Rules not processed for user `3e74ac44e5814122a3226ac023f3688a`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:45.029562 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ea3b073c-e588-405a-9510-ef7c7fde13d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:45.048230 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 820/1642] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Jun 15 19:37:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:37:45.057855 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d028f2-2e66-44f1-82c9-4355e9ec803c None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:45.060043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-21d028f2-2e66-44f1-82c9-4355e9ec803c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:45.171940 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d028f2-2e66-44f1-82c9-4355e9ec803c None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:45.176186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c62f139-c9f7-4a29-a546-2efb6a905ae2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfca4d302aa43249609513f35d55d7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:45.177448 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62f139-c9f7-4a29-a546-2efb6a905ae2 service glance] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:45.178862 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62f139-c9f7-4a29-a546-2efb6a905ae2 service glance] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:45.179244 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62f139-c9f7-4a29-a546-2efb6a905ae2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:45.230929 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c62f139-c9f7-4a29-a546-2efb6a905ae2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:45.239778 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c62f139-c9f7-4a29-a546-2efb6a905ae2 service glance] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:45.275843 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 823/1643] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:37:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:37:45.791584 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6be00b-303f-48f4-adaa-f784c78227b4 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:45.792156 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6be00b-303f-48f4-adaa-f784c78227b4 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:45.792156 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6be00b-303f-48f4-adaa-f784c78227b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:45.852750 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-da6be00b-303f-48f4-adaa-f784c78227b4 None None] MFA Rules not processed for user `97c5493f93ea4e51ac0491a65701845c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:45.854174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-da6be00b-303f-48f4-adaa-f784c78227b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:45.969646 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 821/1644] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:37:45 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:37:45.975823 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c62f139-c9f7-4a29-a546-2efb6a905ae2 service glance] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:45.983561 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:37:46.086939 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322247d-c891-4feb-ad90-eebd20657ad8 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:46.087343 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322247d-c891-4feb-ad90-eebd20657ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:46.087556 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322247d-c891-4feb-ad90-eebd20657ad8 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:46.089199 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 822/1645] 10.222.0.15 () {54 vars in 912 bytes} [Wed Jun 15 19:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 19:37:46.096804 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b9776-7ab4-41c0-b6e5-73c76e79bc77 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:46.097097 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b9776-7ab4-41c0-b6e5-73c76e79bc77 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:46.097380 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b9776-7ab4-41c0-b6e5-73c76e79bc77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:46.114443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c62f139-c9f7-4a29-a546-2efb6a905ae2 service glance] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:46.125124 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57b4a1e-5d28-40d2-a027-20607e834a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:46.125540 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b4a1e-5d28-40d2-a027-20607e834a24 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:46.125721 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b4a1e-5d28-40d2-a027-20607e834a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:46.126000 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b4a1e-5d28-40d2-a027-20607e834a24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:46.127063 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57b4a1e-5d28-40d2-a027-20607e834a24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:46.128977 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57b4a1e-5d28-40d2-a027-20607e834a24 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:46.169933 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-857b9776-7ab4-41c0-b6e5-73c76e79bc77 None None] MFA Rules not processed for user `5c63e0b6ceed4f7cb2e6f9269eb85795`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:46.171271 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-857b9776-7ab4-41c0-b6e5-73c76e79bc77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:46.175864 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 824/1646] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:37:45 2022] POST /identity/v3/projects => generated 405 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:37:46.176873 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57b4a1e-5d28-40d2-a027-20607e834a24 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:46.249303 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57b4a1e-5d28-40d2-a027-20607e834a24 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:46.256250 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36324bd8-2bb1-47a3-b25c-e33c04c12036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:46.257062 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36324bd8-2bb1-47a3-b25c-e33c04c12036 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:46.257513 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36324bd8-2bb1-47a3-b25c-e33c04c12036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:46.258200 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36324bd8-2bb1-47a3-b25c-e33c04c12036 admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:46.259309 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36324bd8-2bb1-47a3-b25c-e33c04c12036 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:46.261131 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36324bd8-2bb1-47a3-b25c-e33c04c12036 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:46.288854 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 823/1647] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Jun 15 19:37:46 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jun 15 19:37:46.290291 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 825/1648] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:37:46 2022] POST /identity/v3/users => generated 309 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:37:46.290941 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b9776-7ab4-41c0-b6e5-73c76e79bc77 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:46.304909 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b9776-7ab4-41c0-b6e5-73c76e79bc77 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:46.311572 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4bc584-2cbd-43d8-8c3a-e0d15f7ac2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:46.312428 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bc584-2cbd-43d8-8c3a-e0d15f7ac2d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:46.312754 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bc584-2cbd-43d8-8c3a-e0d15f7ac2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:46.313106 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bc584-2cbd-43d8-8c3a-e0d15f7ac2d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:46.314331 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4bc584-2cbd-43d8-8c3a-e0d15f7ac2d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:46.316010 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36324bd8-2bb1-47a3-b25c-e33c04c12036 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:46.332460 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4bc584-2cbd-43d8-8c3a-e0d15f7ac2d8 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:46.333631 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36324bd8-2bb1-47a3-b25c-e33c04c12036 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:46.345175 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da06daee-27c1-42b7-a5e4-deabb52a0509 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:46.346904 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06daee-27c1-42b7-a5e4-deabb52a0509 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:46.348221 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06daee-27c1-42b7-a5e4-deabb52a0509 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:46.348331 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06daee-27c1-42b7-a5e4-deabb52a0509 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:46.363017 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 824/1649] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:37:46.364019 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4bc584-2cbd-43d8-8c3a-e0d15f7ac2d8 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:46.367265 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da06daee-27c1-42b7-a5e4-deabb52a0509 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:46.372166 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da06daee-27c1-42b7-a5e4-deabb52a0509 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:46.399919 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4bc584-2cbd-43d8-8c3a-e0d15f7ac2d8 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:46.405239 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 826/1650] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:37:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:37:46.411554 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943bb40f-08ec-455b-b9b8-1370bd24fedd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:46.412571 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943bb40f-08ec-455b-b9b8-1370bd24fedd admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:46.412859 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943bb40f-08ec-455b-b9b8-1370bd24fedd admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:46.413128 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943bb40f-08ec-455b-b9b8-1370bd24fedd admin admin] PATH_INFO: `/v3/projects/74899ee4276c499b8c3a47277d78383c/users/b024bd42b51e4c23ad309d6ccc2181b5/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:46.435421 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:37:46.453332 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943bb40f-08ec-455b-b9b8-1370bd24fedd admin admin] RBAC: Authorizing `identity:create_grant(project_id=74899ee4276c499b8c3a47277d78383c, user_id=b024bd42b51e4c23ad309d6ccc2181b5, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:46.458945 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943bb40f-08ec-455b-b9b8-1370bd24fedd admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:46.490357 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da06daee-27c1-42b7-a5e4-deabb52a0509 service neutron] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:46.490881 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 825/1651] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:37:46 2022] PUT /identity/v3/projects/74899ee4276c499b8c3a47277d78383c/users/b024bd42b51e4c23ad309d6ccc2181b5/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:37:46.507003 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da06daee-27c1-42b7-a5e4-deabb52a0509 service neutron] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:46.682891 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4012d34e-13cf-4a66-9544-688d8f3f8109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:46.683631 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4012d34e-13cf-4a66-9544-688d8f3f8109 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:46.683917 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4012d34e-13cf-4a66-9544-688d8f3f8109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:46.684190 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4012d34e-13cf-4a66-9544-688d8f3f8109 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:46.684941 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4012d34e-13cf-4a66-9544-688d8f3f8109 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:46.686747 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4012d34e-13cf-4a66-9544-688d8f3f8109 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:46.719707 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943bb40f-08ec-455b-b9b8-1370bd24fedd admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:46.734017 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943bb40f-08ec-455b-b9b8-1370bd24fedd admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:46.734734 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 827/1652] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:37:46.746975 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376f305-f0f6-4898-ab88-d6f874f894f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:46.747904 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376f305-f0f6-4898-ab88-d6f874f894f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:46.748231 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376f305-f0f6-4898-ab88-d6f874f894f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:46.748512 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376f305-f0f6-4898-ab88-d6f874f894f8 admin admin] PATH_INFO: `/v3/projects/74899ee4276c499b8c3a47277d78383c/users/b024bd42b51e4c23ad309d6ccc2181b5/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:46.754894 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e376f305-f0f6-4898-ab88-d6f874f894f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74899ee4276c499b8c3a47277d78383c, user_id=b024bd42b51e4c23ad309d6ccc2181b5, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:46.757696 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e376f305-f0f6-4898-ab88-d6f874f894f8 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:46.784882 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 826/1653] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:37:46 2022] PUT /identity/v3/projects/74899ee4276c499b8c3a47277d78383c/users/b024bd42b51e4c23ad309d6ccc2181b5/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:37:47.929143 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6972dce-cccc-4c84-94da-b8dca03a885d None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:47.929664 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6972dce-cccc-4c84-94da-b8dca03a885d None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:47.929748 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6972dce-cccc-4c84-94da-b8dca03a885d None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:47.931349 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 828/1654] 10.222.0.15 () {54 vars in 960 bytes} [Wed Jun 15 19:37:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:37:47.937346 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07bb072-c3fe-4cd1-8804-b40d8068a9d3 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:47.937862 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07bb072-c3fe-4cd1-8804-b40d8068a9d3 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:47.937944 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07bb072-c3fe-4cd1-8804-b40d8068a9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:47.992183 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c07bb072-c3fe-4cd1-8804-b40d8068a9d3 None None] MFA Rules not processed for user `cb0ec2ef6c4748e181f1963e7dfe30b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:47.993577 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c07bb072-c3fe-4cd1-8804-b40d8068a9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:48.122487 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 827/1655] 10.222.0.15 () {58 vars in 1057 bytes} [Wed Jun 15 19:37:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:37:48.131441 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6972dce-cccc-4c84-94da-b8dca03a885d None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:48.139582 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:37:48.160009 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6972dce-cccc-4c84-94da-b8dca03a885d None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:48.171916 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a9957-fe20-44f6-94f2-b256120ad4fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb0ec2ef6c4748e181f1963e7dfe30b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:48.171916 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a9957-fe20-44f6-94f2-b256120ad4fd service placement] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:48.171916 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a9957-fe20-44f6-94f2-b256120ad4fd service placement] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:48.171916 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a9957-fe20-44f6-94f2-b256120ad4fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:48.178151 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-055a9957-fe20-44f6-94f2-b256120ad4fd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:48.224049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a9957-fe20-44f6-94f2-b256120ad4fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:48.225887 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a9957-fe20-44f6-94f2-b256120ad4fd service placement] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:48.251882 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 829/1656] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:37:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:37:48.295633 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30794f16-bf01-47f9-9058-f7eab41d352c None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:48.295932 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30794f16-bf01-47f9-9058-f7eab41d352c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:48.296216 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30794f16-bf01-47f9-9058-f7eab41d352c None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:48.298115 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 828/1657] 10.222.0.15 () {54 vars in 961 bytes} [Wed Jun 15 19:37:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:37:48.318507 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f47ee-e401-433d-90e6-e2a4fb433639 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:48.318797 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f47ee-e401-433d-90e6-e2a4fb433639 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:48.319094 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f47ee-e401-433d-90e6-e2a4fb433639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:48.355830 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-871f47ee-e401-433d-90e6-e2a4fb433639 None None] MFA Rules not processed for user `3e74ac44e5814122a3226ac023f3688a`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:48.357120 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-871f47ee-e401-433d-90e6-e2a4fb433639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:48.521224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 830/1658] 10.222.0.15 () {58 vars in 1058 bytes} [Wed Jun 15 19:37:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:37:48.527590 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89033f-0fa7-43bd-8447-afb069ecaf1f None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:48.527881 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89033f-0fa7-43bd-8447-afb069ecaf1f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:48.528147 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89033f-0fa7-43bd-8447-afb069ecaf1f None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:48.529851 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 829/1659] 10.222.0.15 () {54 vars in 912 bytes} [Wed Jun 15 19:37:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:37:48.535943 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65b059-a450-4b36-ae7f-974e6eeb4c9a None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:48.536237 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65b059-a450-4b36-ae7f-974e6eeb4c9a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:48.536535 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65b059-a450-4b36-ae7f-974e6eeb4c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:48.576187 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-5b65b059-a450-4b36-ae7f-974e6eeb4c9a None None] MFA Rules not processed for user `3e74ac44e5814122a3226ac023f3688a`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:48.577615 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-5b65b059-a450-4b36-ae7f-974e6eeb4c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:48.610435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 831/1660] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Jun 15 19:37:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:37:48.626551 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89033f-0fa7-43bd-8447-afb069ecaf1f None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:48.744703 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89033f-0fa7-43bd-8447-afb069ecaf1f None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:48.754381 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d206900-c5e5-44d9-b269-aad46c255a5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:48.755060 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d206900-c5e5-44d9-b269-aad46c255a5e service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:48.755346 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d206900-c5e5-44d9-b269-aad46c255a5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:48.756834 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d206900-c5e5-44d9-b269-aad46c255a5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:48.785733 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d206900-c5e5-44d9-b269-aad46c255a5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:48.787241 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d206900-c5e5-44d9-b269-aad46c255a5e service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:48.815493 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 830/1661] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:37:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:37:51.383477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a9607-b9e4-41df-8a99-4c8daff83a7c None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:51.384370 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a9607-b9e4-41df-8a99-4c8daff83a7c None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:51.384840 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a9607-b9e4-41df-8a99-4c8daff83a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:51.442466 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-6e2a9607-b9e4-41df-8a99-4c8daff83a7c None None] MFA Rules not processed for user `b024bd42b51e4c23ad309d6ccc2181b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:51.443847 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-6e2a9607-b9e4-41df-8a99-4c8daff83a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:51.601674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 832/1662] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:37:51 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:37:54.245340 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb6e73-ab50-49ce-8861-fd3a4f5733d9 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:54.245861 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb6e73-ab50-49ce-8861-fd3a4f5733d9 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:54.245934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb6e73-ab50-49ce-8861-fd3a4f5733d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:54.265980 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-adbb6e73-ab50-49ce-8861-fd3a4f5733d9 None None] MFA Rules not processed for user `67aa048d409c46e9a729aec9d1b2286e`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:54.267562 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-adbb6e73-ab50-49ce-8861-fd3a4f5733d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:54.385286 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 831/1663] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Jun 15 19:37:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:37:54.467156 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a981b3-db22-4313-a548-b723ec622e84 None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:54.467691 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a981b3-db22-4313-a548-b723ec622e84 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:54.467761 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a981b3-db22-4313-a548-b723ec622e84 None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:54.469818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 833/1664] 10.222.0.15 () {54 vars in 921 bytes} [Wed Jun 15 19:37:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:37:54.495805 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8fa26-698c-478a-b3aa-7f284daf22e8 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:54.496243 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8fa26-698c-478a-b3aa-7f284daf22e8 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:54.496626 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8fa26-698c-478a-b3aa-7f284daf22e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:54.539905 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-44e8fa26-698c-478a-b3aa-7f284daf22e8 None None] MFA Rules not processed for user `cb0ec2ef6c4748e181f1963e7dfe30b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:54.547418 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-44e8fa26-698c-478a-b3aa-7f284daf22e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:54.577080 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 832/1665] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Jun 15 19:37:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:37:54.624045 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a981b3-db22-4313-a548-b723ec622e84 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:54.650077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a981b3-db22-4313-a548-b723ec622e84 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:54.664993 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ed71c-07f2-4a1b-ac64-8e5362b19e5e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb0ec2ef6c4748e181f1963e7dfe30b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:54.665984 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ed71c-07f2-4a1b-ac64-8e5362b19e5e service placement] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:54.671396 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ed71c-07f2-4a1b-ac64-8e5362b19e5e service placement] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:54.671926 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ed71c-07f2-4a1b-ac64-8e5362b19e5e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:54.690569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ed71c-07f2-4a1b-ac64-8e5362b19e5e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:54.693895 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e8fa26-698c-478a-b3aa-7f284daf22e8 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:54.696051 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-44e8fa26-698c-478a-b3aa-7f284daf22e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:54.710536 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ed71c-07f2-4a1b-ac64-8e5362b19e5e service placement] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:54.724453 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:37:54.774211 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 834/1666] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:37:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:37:54.892742 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e8fa26-698c-478a-b3aa-7f284daf22e8 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:54.907493 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5de09db-78c6-455e-b22c-f9775e23b68f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176d5775a70f4524a9b7fe1ce9d426ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:54.908224 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de09db-78c6-455e-b22c-f9775e23b68f service swift] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:54.908511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de09db-78c6-455e-b22c-f9775e23b68f service swift] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:54.908787 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de09db-78c6-455e-b22c-f9775e23b68f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:54.935181 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5de09db-78c6-455e-b22c-f9775e23b68f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:54.937356 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5de09db-78c6-455e-b22c-f9775e23b68f service swift] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:54.953926 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 833/1667] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Jun 15 19:37:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 15 19:37:55.818715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10a05b-1494-47b9-8b86-b41826f57ad2 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:55.819267 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10a05b-1494-47b9-8b86-b41826f57ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:55.819267 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10a05b-1494-47b9-8b86-b41826f57ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:55.924240 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-ec10a05b-1494-47b9-8b86-b41826f57ad2 None None] MFA Rules not processed for user `c815423179f44b219e4f8572b10355c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:55.928199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-ec10a05b-1494-47b9-8b86-b41826f57ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:56.132291 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:37:56.334866 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 835/1668] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:37:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 517 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:37:56.342602 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5de09db-78c6-455e-b22c-f9775e23b68f service swift] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:56.378078 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5de09db-78c6-455e-b22c-f9775e23b68f service swift] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:56.413563 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b6cdf-a556-42c1-a3b3-80a5faf293c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:56.416217 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b6cdf-a556-42c1-a3b3-80a5faf293c6 service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:56.418036 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b6cdf-a556-42c1-a3b3-80a5faf293c6 service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:56.418494 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b6cdf-a556-42c1-a3b3-80a5faf293c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:56.454279 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03b6cdf-a556-42c1-a3b3-80a5faf293c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:56.466361 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03b6cdf-a556-42c1-a3b3-80a5faf293c6 service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:56.522802 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 834/1669] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:37:56 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:37:57.355106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10a05b-1494-47b9-8b86-b41826f57ad2 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:57.525588 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10a05b-1494-47b9-8b86-b41826f57ad2 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:57.546321 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca45fdc-2d5d-4e86-a827-8e0b117ac0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:57.549941 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca45fdc-2d5d-4e86-a827-8e0b117ac0db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:57.550503 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca45fdc-2d5d-4e86-a827-8e0b117ac0db admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:57.552198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca45fdc-2d5d-4e86-a827-8e0b117ac0db admin admin] PATH_INFO: `/v3/users/97c5493f93ea4e51ac0491a65701845c` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:57.556946 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca45fdc-2d5d-4e86-a827-8e0b117ac0db admin admin] RBAC: Authorizing `identity:delete_user(user_id=97c5493f93ea4e51ac0491a65701845c)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:57.560004 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: WARNING py.warnings [None req-7ca45fdc-2d5d-4e86-a827-8e0b117ac0db admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 19:37:57.560004 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: warnings.warn(msg) Jun 15 19:37:57.560004 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: Jun 15 19:37:57.564229 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca45fdc-2d5d-4e86-a827-8e0b117ac0db admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:57.739673 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-7ca45fdc-2d5d-4e86-a827-8e0b117ac0db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97c5493f93ea4e51ac0491a65701845c'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:37:57.741027 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-7ca45fdc-2d5d-4e86-a827-8e0b117ac0db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97c5493f93ea4e51ac0491a65701845c'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:37:57.760227 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-7ca45fdc-2d5d-4e86-a827-8e0b117ac0db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97c5493f93ea4e51ac0491a65701845c'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:37:57.763077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-7ca45fdc-2d5d-4e86-a827-8e0b117ac0db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97c5493f93ea4e51ac0491a65701845c'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:37:57.786492 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-7ca45fdc-2d5d-4e86-a827-8e0b117ac0db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97c5493f93ea4e51ac0491a65701845c'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:37:57.928710 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 836/1670] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:37:57 2022] DELETE /identity/v3/users/97c5493f93ea4e51ac0491a65701845c => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:37:58.264704 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b6cdf-a556-42c1-a3b3-80a5faf293c6 service nova] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:58.285346 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b6cdf-a556-42c1-a3b3-80a5faf293c6 service nova] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:58.463813 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1c83fe-14b9-40f1-9199-a1b0417e8a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:58.464902 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c83fe-14b9-40f1-9199-a1b0417e8a69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:58.465435 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c83fe-14b9-40f1-9199-a1b0417e8a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:58.466322 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c83fe-14b9-40f1-9199-a1b0417e8a69 admin admin] PATH_INFO: `/v3/users/1a23e828921f4344bcef652c6d4a5db6` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:58.468415 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1c83fe-14b9-40f1-9199-a1b0417e8a69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a23e828921f4344bcef652c6d4a5db6)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:58.472265 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1c83fe-14b9-40f1-9199-a1b0417e8a69 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:58.673916 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2a1c83fe-14b9-40f1-9199-a1b0417e8a69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a23e828921f4344bcef652c6d4a5db6'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:37:58.675997 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2a1c83fe-14b9-40f1-9199-a1b0417e8a69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a23e828921f4344bcef652c6d4a5db6'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:37:58.677067 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2a1c83fe-14b9-40f1-9199-a1b0417e8a69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a23e828921f4344bcef652c6d4a5db6'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:37:58.751972 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2a1c83fe-14b9-40f1-9199-a1b0417e8a69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a23e828921f4344bcef652c6d4a5db6'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:37:58.798352 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2a1c83fe-14b9-40f1-9199-a1b0417e8a69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a23e828921f4344bcef652c6d4a5db6'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:37:58.805579 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca45fdc-2d5d-4e86-a827-8e0b117ac0db admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:58.808279 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-7ca45fdc-2d5d-4e86-a827-8e0b117ac0db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:58.851387 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca45fdc-2d5d-4e86-a827-8e0b117ac0db admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:58.860076 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 835/1671] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:37:58 2022] DELETE /identity/v3/users/1a23e828921f4344bcef652c6d4a5db6 => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:37:58.862077 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba5a1b-df4e-4932-969a-fdcca4e53654 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:58.871284 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba5a1b-df4e-4932-969a-fdcca4e53654 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:58.871829 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba5a1b-df4e-4932-969a-fdcca4e53654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:58.918669 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:37:58.936891 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-fbba5a1b-df4e-4932-969a-fdcca4e53654 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:37:58.944670 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-fbba5a1b-df4e-4932-969a-fdcca4e53654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:59.053644 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:37:59.084019 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a5d70-9116-42f2-a4db-206f667014e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:59.086553 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a5d70-9116-42f2-a4db-206f667014e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:59.088483 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a5d70-9116-42f2-a4db-206f667014e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:59.089356 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a5d70-9116-42f2-a4db-206f667014e5 admin admin] PATH_INFO: `/v3/projects/b392b3914e7f4a1b93bbfc0f3ab8cfc1` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:59.093100 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a5d70-9116-42f2-a4db-206f667014e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b392b3914e7f4a1b93bbfc0f3ab8cfc1)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:59.097577 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a5d70-9116-42f2-a4db-206f667014e5 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:59.146599 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 836/1672] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:37:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:37:59.217278 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-ed2a5d70-9116-42f2-a4db-206f667014e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b392b3914e7f4a1b93bbfc0f3ab8cfc1', 'request_id': 'req-ed2a5d70-9116-42f2-a4db-206f667014e5'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:37:59.235334 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-ed2a5d70-9116-42f2-a4db-206f667014e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b392b3914e7f4a1b93bbfc0f3ab8cfc1', 'request_id': 'req-ed2a5d70-9116-42f2-a4db-206f667014e5'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:37:59.252109 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-ed2a5d70-9116-42f2-a4db-206f667014e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b392b3914e7f4a1b93bbfc0f3ab8cfc1', 'request_id': 'req-ed2a5d70-9116-42f2-a4db-206f667014e5'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:37:59.264918 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba5a1b-df4e-4932-969a-fdcca4e53654 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:59.266652 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-fbba5a1b-df4e-4932-969a-fdcca4e53654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:59.304767 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-ed2a5d70-9116-42f2-a4db-206f667014e5 admin admin] The token cache is being invalidate because project b392b3914e7f4a1b93bbfc0f3ab8cfc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:37:59.306497 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-ed2a5d70-9116-42f2-a4db-206f667014e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:37:59.308035 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-ed2a5d70-9116-42f2-a4db-206f667014e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:37:59.311695 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 837/1673] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:37:58 2022] DELETE /identity/v3/projects/b392b3914e7f4a1b93bbfc0f3ab8cfc1 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:37:59.319555 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a5d70-9116-42f2-a4db-206f667014e5 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:37:59.330958 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-ed2a5d70-9116-42f2-a4db-206f667014e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:59.503407 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba5a1b-df4e-4932-969a-fdcca4e53654 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:59.610246 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a5d70-9116-42f2-a4db-206f667014e5 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:37:59.621092 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597861bc-21c3-4576-b17d-2e780c38ca40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:59.622069 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597861bc-21c3-4576-b17d-2e780c38ca40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:59.622374 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597861bc-21c3-4576-b17d-2e780c38ca40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:59.622686 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597861bc-21c3-4576-b17d-2e780c38ca40 admin admin] PATH_INFO: `/v3/users/b024bd42b51e4c23ad309d6ccc2181b5` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:59.626826 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597861bc-21c3-4576-b17d-2e780c38ca40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b024bd42b51e4c23ad309d6ccc2181b5)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:59.632256 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597861bc-21c3-4576-b17d-2e780c38ca40 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:59.649718 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff9a7f-7311-4b63-9db2-2fbc78b5d009 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:37:59.650565 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff9a7f-7311-4b63-9db2-2fbc78b5d009 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:37:59.650962 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff9a7f-7311-4b63-9db2-2fbc78b5d009 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:37:59.652646 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff9a7f-7311-4b63-9db2-2fbc78b5d009 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:37:59.660287 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1eff9a7f-7311-4b63-9db2-2fbc78b5d009 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:37:59.702692 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eff9a7f-7311-4b63-9db2-2fbc78b5d009 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:37:59.706511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eff9a7f-7311-4b63-9db2-2fbc78b5d009 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:37:59.755341 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 837/1674] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:37:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:37:59.803063 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-597861bc-21c3-4576-b17d-2e780c38ca40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b024bd42b51e4c23ad309d6ccc2181b5'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:37:59.807061 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-597861bc-21c3-4576-b17d-2e780c38ca40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b024bd42b51e4c23ad309d6ccc2181b5'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:37:59.830507 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-597861bc-21c3-4576-b17d-2e780c38ca40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b024bd42b51e4c23ad309d6ccc2181b5'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:37:59.832406 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-597861bc-21c3-4576-b17d-2e780c38ca40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b024bd42b51e4c23ad309d6ccc2181b5'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:37:59.849357 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-597861bc-21c3-4576-b17d-2e780c38ca40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b024bd42b51e4c23ad309d6ccc2181b5'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:37:59.901571 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 838/1675] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:37:59 2022] DELETE /identity/v3/users/b024bd42b51e4c23ad309d6ccc2181b5 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:00.462341 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff9a7f-7311-4b63-9db2-2fbc78b5d009 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:00.475776 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff9a7f-7311-4b63-9db2-2fbc78b5d009 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:00.611611 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888f1d54-832e-426c-b713-840c46bbb6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:00.612656 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f1d54-832e-426c-b713-840c46bbb6ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:00.613073 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f1d54-832e-426c-b713-840c46bbb6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:00.613480 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f1d54-832e-426c-b713-840c46bbb6ad admin admin] PATH_INFO: `/v3/projects/74899ee4276c499b8c3a47277d78383c` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:00.614888 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888f1d54-832e-426c-b713-840c46bbb6ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=74899ee4276c499b8c3a47277d78383c)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:00.616078 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: WARNING py.warnings [None req-888f1d54-832e-426c-b713-840c46bbb6ad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 19:38:00.616078 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: warnings.warn(msg) Jun 15 19:38:00.616078 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: Jun 15 19:38:00.617567 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888f1d54-832e-426c-b713-840c46bbb6ad admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:00.627230 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:00.703706 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-888f1d54-832e-426c-b713-840c46bbb6ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74899ee4276c499b8c3a47277d78383c', 'request_id': 'req-888f1d54-832e-426c-b713-840c46bbb6ad'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:00.715193 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-888f1d54-832e-426c-b713-840c46bbb6ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74899ee4276c499b8c3a47277d78383c', 'request_id': 'req-888f1d54-832e-426c-b713-840c46bbb6ad'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:00.740215 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-888f1d54-832e-426c-b713-840c46bbb6ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74899ee4276c499b8c3a47277d78383c', 'request_id': 'req-888f1d54-832e-426c-b713-840c46bbb6ad'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:00.762331 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-888f1d54-832e-426c-b713-840c46bbb6ad admin admin] The token cache is being invalidate because project 74899ee4276c499b8c3a47277d78383c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:00.762962 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-888f1d54-832e-426c-b713-840c46bbb6ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:00.764885 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-888f1d54-832e-426c-b713-840c46bbb6ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:00.784898 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991d6e4-1b38-4bb7-907f-fd145c6a0b06 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:00.784898 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991d6e4-1b38-4bb7-907f-fd145c6a0b06 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:00.784898 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991d6e4-1b38-4bb7-907f-fd145c6a0b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:00.787379 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 838/1676] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:00 2022] DELETE /identity/v3/projects/74899ee4276c499b8c3a47277d78383c => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:00.817156 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:00.855697 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-8991d6e4-1b38-4bb7-907f-fd145c6a0b06 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:00.859728 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-8991d6e4-1b38-4bb7-907f-fd145c6a0b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:00.890837 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22873cf-928e-4027-a118-69dbf4652fd2 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:00.891139 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22873cf-928e-4027-a118-69dbf4652fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:00.891476 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22873cf-928e-4027-a118-69dbf4652fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:01.006345 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-f22873cf-928e-4027-a118-69dbf4652fd2 None None] MFA Rules not processed for user `b2103a7ed6824444a60fa5573e7ad39e`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:01.012327 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f22873cf-928e-4027-a118-69dbf4652fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:01.154336 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 839/1677] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:01.161648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22efa24-c7f5-4a08-994c-96cd4863eac4 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:01.164724 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22efa24-c7f5-4a08-994c-96cd4863eac4 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:01.165440 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22efa24-c7f5-4a08-994c-96cd4863eac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:01.221099 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-f22efa24-c7f5-4a08-994c-96cd4863eac4 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:01.224926 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-f22efa24-c7f5-4a08-994c-96cd4863eac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:01.251681 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 840/1678] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:38:01.256865 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22efa24-c7f5-4a08-994c-96cd4863eac4 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:01.265285 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 839/1679] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:01.281097 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22873cf-928e-4027-a118-69dbf4652fd2 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:01.284851 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f22873cf-928e-4027-a118-69dbf4652fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:01.289647 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22efa24-c7f5-4a08-994c-96cd4863eac4 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:01.303999 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec4a133-473d-45e4-b7a4-10e0797b81fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:01.306327 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec4a133-473d-45e4-b7a4-10e0797b81fa admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:01.306844 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec4a133-473d-45e4-b7a4-10e0797b81fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:01.309711 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec4a133-473d-45e4-b7a4-10e0797b81fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:01.310864 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec4a133-473d-45e4-b7a4-10e0797b81fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:38:01.311994 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec4a133-473d-45e4-b7a4-10e0797b81fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:01.314354 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec4a133-473d-45e4-b7a4-10e0797b81fa admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:01.372068 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 841/1680] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:38:01.377388 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec4a133-473d-45e4-b7a4-10e0797b81fa admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:01.400389 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec4a133-473d-45e4-b7a4-10e0797b81fa admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:01.416850 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cee91c-254a-470d-a8c4-1bee74f9d00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:01.421443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cee91c-254a-470d-a8c4-1bee74f9d00d admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:01.421443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cee91c-254a-470d-a8c4-1bee74f9d00d admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:01.421443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cee91c-254a-470d-a8c4-1bee74f9d00d admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:01.422520 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cee91c-254a-470d-a8c4-1bee74f9d00d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:01.426982 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cee91c-254a-470d-a8c4-1bee74f9d00d admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:01.540359 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 842/1681] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:01 2022] POST /identity/v3/projects => generated 369 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:01.541648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cee91c-254a-470d-a8c4-1bee74f9d00d admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:01.545906 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-77cee91c-254a-470d-a8c4-1bee74f9d00d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:01.564099 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22873cf-928e-4027-a118-69dbf4652fd2 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:01.611054 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:01.718730 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1243d9a1-95b9-453c-8247-7010105efb61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:01.719430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1243d9a1-95b9-453c-8247-7010105efb61 service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:01.719715 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1243d9a1-95b9-453c-8247-7010105efb61 service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:01.723151 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1243d9a1-95b9-453c-8247-7010105efb61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:01.775469 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cee91c-254a-470d-a8c4-1bee74f9d00d admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:01.790641 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde7ca23-a4de-4738-bf06-756a8a4f1be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:01.797813 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde7ca23-a4de-4738-bf06-756a8a4f1be3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:01.798479 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde7ca23-a4de-4738-bf06-756a8a4f1be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:01.798687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde7ca23-a4de-4738-bf06-756a8a4f1be3 admin admin] PATH_INFO: `/v3/projects/03f70bdddb8d4e808b0eade72feaadda` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:01.802560 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde7ca23-a4de-4738-bf06-756a8a4f1be3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03f70bdddb8d4e808b0eade72feaadda)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:01.805464 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde7ca23-a4de-4738-bf06-756a8a4f1be3 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:01.816110 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1243d9a1-95b9-453c-8247-7010105efb61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:01.818346 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1243d9a1-95b9-453c-8247-7010105efb61 service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:01.849690 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 840/1682] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:01 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:38:01.851209 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1243d9a1-95b9-453c-8247-7010105efb61 service nova] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:01.868700 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1243d9a1-95b9-453c-8247-7010105efb61 service nova] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:01.892498 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c254740-e82d-4a97-9a54-6c42dfd8bfa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:01.893341 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c254740-e82d-4a97-9a54-6c42dfd8bfa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:01.898769 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c254740-e82d-4a97-9a54-6c42dfd8bfa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:01.901289 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c254740-e82d-4a97-9a54-6c42dfd8bfa5 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:01.902600 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c254740-e82d-4a97-9a54-6c42dfd8bfa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:01.917851 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c254740-e82d-4a97-9a54-6c42dfd8bfa5 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:01.991443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-cde7ca23-a4de-4738-bf06-756a8a4f1be3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03f70bdddb8d4e808b0eade72feaadda', 'request_id': 'req-cde7ca23-a4de-4738-bf06-756a8a4f1be3'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:02.011854 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 841/1683] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:01 2022] POST /identity/v3/users => generated 285 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:02.014943 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c254740-e82d-4a97-9a54-6c42dfd8bfa5 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:02.018574 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-cde7ca23-a4de-4738-bf06-756a8a4f1be3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03f70bdddb8d4e808b0eade72feaadda', 'request_id': 'req-cde7ca23-a4de-4738-bf06-756a8a4f1be3'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:02.058449 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-cde7ca23-a4de-4738-bf06-756a8a4f1be3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03f70bdddb8d4e808b0eade72feaadda', 'request_id': 'req-cde7ca23-a4de-4738-bf06-756a8a4f1be3'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:02.065519 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c254740-e82d-4a97-9a54-6c42dfd8bfa5 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:02.080658 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:02.114171 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-cde7ca23-a4de-4738-bf06-756a8a4f1be3 admin admin] The token cache is being invalidate because project 03f70bdddb8d4e808b0eade72feaadda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:02.114587 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-cde7ca23-a4de-4738-bf06-756a8a4f1be3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:02.115434 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-cde7ca23-a4de-4738-bf06-756a8a4f1be3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:02.118000 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 843/1684] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:01 2022] DELETE /identity/v3/projects/03f70bdddb8d4e808b0eade72feaadda => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:02.122643 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde7ca23-a4de-4738-bf06-756a8a4f1be3 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:02.125039 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-cde7ca23-a4de-4738-bf06-756a8a4f1be3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:02.219492 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d55b3-df87-46f6-b3a6-6a84fab7c8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:02.220451 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d55b3-df87-46f6-b3a6-6a84fab7c8d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:02.220729 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d55b3-df87-46f6-b3a6-6a84fab7c8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:02.221007 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d55b3-df87-46f6-b3a6-6a84fab7c8d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:02.221836 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632d55b3-df87-46f6-b3a6-6a84fab7c8d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:02.223815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632d55b3-df87-46f6-b3a6-6a84fab7c8d1 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:02.238160 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 842/1685] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:02 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:02.241368 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d55b3-df87-46f6-b3a6-6a84fab7c8d1 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:02.243221 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-632d55b3-df87-46f6-b3a6-6a84fab7c8d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:02.257813 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d55b3-df87-46f6-b3a6-6a84fab7c8d1 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:02.262971 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8235cca7-3311-4fca-9dd1-472fcb85848b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:02.264193 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235cca7-3311-4fca-9dd1-472fcb85848b admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:02.264518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235cca7-3311-4fca-9dd1-472fcb85848b admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:02.264764 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235cca7-3311-4fca-9dd1-472fcb85848b admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:02.265604 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8235cca7-3311-4fca-9dd1-472fcb85848b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:02.267653 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8235cca7-3311-4fca-9dd1-472fcb85848b admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:02.300915 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 843/1686] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:02 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:02.301850 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8235cca7-3311-4fca-9dd1-472fcb85848b admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:02.329346 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8235cca7-3311-4fca-9dd1-472fcb85848b admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:02.342146 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad70b2d6-8fc2-47f5-a783-6eca90eacdc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:02.343146 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad70b2d6-8fc2-47f5-a783-6eca90eacdc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:02.344545 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad70b2d6-8fc2-47f5-a783-6eca90eacdc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:02.344767 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad70b2d6-8fc2-47f5-a783-6eca90eacdc4 admin admin] PATH_INFO: `/v3/projects/2305125818b54552bc09b0942be3e4d5/users/c1786f9a1ccb4eff8a8deab64dd6e1dd/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:02.368639 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad70b2d6-8fc2-47f5-a783-6eca90eacdc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2305125818b54552bc09b0942be3e4d5, user_id=c1786f9a1ccb4eff8a8deab64dd6e1dd, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:02.393674 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad70b2d6-8fc2-47f5-a783-6eca90eacdc4 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:02.418764 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 844/1687] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:02 2022] PUT /identity/v3/projects/2305125818b54552bc09b0942be3e4d5/users/c1786f9a1ccb4eff8a8deab64dd6e1dd/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:02.430305 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded80a92-1675-467c-9626-8a7fc2107855 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:02.430674 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded80a92-1675-467c-9626-8a7fc2107855 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:02.431242 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded80a92-1675-467c-9626-8a7fc2107855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:02.441554 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:02.480242 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-ded80a92-1675-467c-9626-8a7fc2107855 None None] MFA Rules not processed for user `c1786f9a1ccb4eff8a8deab64dd6e1dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:02.481705 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ded80a92-1675-467c-9626-8a7fc2107855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:02.503341 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde7ca23-a4de-4738-bf06-756a8a4f1be3 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:02.577566 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 845/1688] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:02 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:02.579044 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded80a92-1675-467c-9626-8a7fc2107855 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:02.684122 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded80a92-1675-467c-9626-8a7fc2107855 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:02.687026 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a04bc-6e68-4726-b136-660724ee6bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:02.691115 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b515178-47ad-499a-9aff-5ed254da94a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:02.692374 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b515178-47ad-499a-9aff-5ed254da94a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:02.693082 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b515178-47ad-499a-9aff-5ed254da94a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:02.693401 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a04bc-6e68-4726-b136-660724ee6bef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:02.693508 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b515178-47ad-499a-9aff-5ed254da94a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:02.693899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a04bc-6e68-4726-b136-660724ee6bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:02.694338 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a04bc-6e68-4726-b136-660724ee6bef admin admin] PATH_INFO: `/v3/users/63d4354f0e63436abaac2e27942f0cab` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:02.695222 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b515178-47ad-499a-9aff-5ed254da94a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:02.697365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a04bc-6e68-4726-b136-660724ee6bef admin admin] RBAC: Authorizing `identity:delete_user(user_id=63d4354f0e63436abaac2e27942f0cab)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:02.697483 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b515178-47ad-499a-9aff-5ed254da94a5 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:02.699583 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a04bc-6e68-4726-b136-660724ee6bef admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:02.749903 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 846/1689] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:02 2022] POST /identity/v3/projects => generated 369 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:02.757641 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b515178-47ad-499a-9aff-5ed254da94a5 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:02.865328 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-2b7a04bc-6e68-4726-b136-660724ee6bef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63d4354f0e63436abaac2e27942f0cab'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:02.868818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-2b7a04bc-6e68-4726-b136-660724ee6bef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63d4354f0e63436abaac2e27942f0cab'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:02.928701 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-2b7a04bc-6e68-4726-b136-660724ee6bef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63d4354f0e63436abaac2e27942f0cab'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:02.931905 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-2b7a04bc-6e68-4726-b136-660724ee6bef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63d4354f0e63436abaac2e27942f0cab'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:02.957249 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-2b7a04bc-6e68-4726-b136-660724ee6bef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63d4354f0e63436abaac2e27942f0cab'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:02.977955 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b515178-47ad-499a-9aff-5ed254da94a5 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:02.981606 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 844/1690] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:38:02 2022] DELETE /identity/v3/users/63d4354f0e63436abaac2e27942f0cab => generated 0 bytes in 860 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:02.983796 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef74070-8d8e-43d2-867d-b6f3f38efe0c None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:02.984293 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef74070-8d8e-43d2-867d-b6f3f38efe0c None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:02.984622 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57291f-4596-4ad9-adaa-31a10c261253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:02.984888 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef74070-8d8e-43d2-867d-b6f3f38efe0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:02.985550 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57291f-4596-4ad9-adaa-31a10c261253 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:02.985946 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57291f-4596-4ad9-adaa-31a10c261253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:02.986349 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57291f-4596-4ad9-adaa-31a10c261253 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:02.987299 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f57291f-4596-4ad9-adaa-31a10c261253 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:02.990120 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f57291f-4596-4ad9-adaa-31a10c261253 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:03.026731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 847/1691] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:02 2022] POST /identity/v3/users => generated 291 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:03.028138 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57291f-4596-4ad9-adaa-31a10c261253 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:03.030475 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-5ef74070-8d8e-43d2-867d-b6f3f38efe0c None None] MFA Rules not processed for user `5c63e0b6ceed4f7cb2e6f9269eb85795`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:03.030960 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7f57291f-4596-4ad9-adaa-31a10c261253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:03.039369 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-5ef74070-8d8e-43d2-867d-b6f3f38efe0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:03.047842 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57291f-4596-4ad9-adaa-31a10c261253 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:03.057201 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:03.101250 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:03.227384 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 845/1692] 10.222.0.42 () {58 vars in 1015 bytes} [Wed Jun 15 19:38:02 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:03.239648 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b2b065-2e6a-48ad-a59c-f7734ee003c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:03.242467 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b2b065-2e6a-48ad-a59c-f7734ee003c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:03.243066 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b2b065-2e6a-48ad-a59c-f7734ee003c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:03.243628 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b2b065-2e6a-48ad-a59c-f7734ee003c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:03.246004 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b2b065-2e6a-48ad-a59c-f7734ee003c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:03.247033 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9c1bb-e0d6-42de-877f-e88c1af45db2 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:03.247277 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9c1bb-e0d6-42de-877f-e88c1af45db2 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:03.255285 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9c1bb-e0d6-42de-877f-e88c1af45db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:03.260774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b2b065-2e6a-48ad-a59c-f7734ee003c6 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:03.301099 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 848/1693] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:03.306487 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b2b065-2e6a-48ad-a59c-f7734ee003c6 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:03.308528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-24b2b065-2e6a-48ad-a59c-f7734ee003c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:03.347154 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-91a9c1bb-e0d6-42de-877f-e88c1af45db2 None None] MFA Rules not processed for user `f26a832b69564f2a8387215afba6730e`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:03.348385 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-91a9c1bb-e0d6-42de-877f-e88c1af45db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:03.367730 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b2b065-2e6a-48ad-a59c-f7734ee003c6 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:03.378248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc926cd6-8104-4602-807b-d4e26b4f7ce7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:03.382866 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc926cd6-8104-4602-807b-d4e26b4f7ce7 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:03.383701 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc926cd6-8104-4602-807b-d4e26b4f7ce7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:03.384828 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc926cd6-8104-4602-807b-d4e26b4f7ce7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:03.418731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc926cd6-8104-4602-807b-d4e26b4f7ce7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:03.423229 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc926cd6-8104-4602-807b-d4e26b4f7ce7 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:03.443864 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 849/1694] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:38:03.444905 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc926cd6-8104-4602-807b-d4e26b4f7ce7 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:03.471887 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc926cd6-8104-4602-807b-d4e26b4f7ce7 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:03.482790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e934e4-34ed-4a05-87dc-d786b09e4ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:03.484557 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e934e4-34ed-4a05-87dc-d786b09e4ffb admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:03.485056 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e934e4-34ed-4a05-87dc-d786b09e4ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:03.485554 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e934e4-34ed-4a05-87dc-d786b09e4ffb admin admin] PATH_INFO: `/v3/projects/3b6fdee1681f41e388d25b044414b360/users/c135e6837b7b4d02963e5ae2e81c1aff/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:03.508679 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e934e4-34ed-4a05-87dc-d786b09e4ffb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b6fdee1681f41e388d25b044414b360, user_id=c135e6837b7b4d02963e5ae2e81c1aff, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:03.514896 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e934e4-34ed-4a05-87dc-d786b09e4ffb admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:03.540383 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 850/1695] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:03 2022] PUT /identity/v3/projects/3b6fdee1681f41e388d25b044414b360/users/c135e6837b7b4d02963e5ae2e81c1aff/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:03.545477 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e934e4-34ed-4a05-87dc-d786b09e4ffb admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:03.674988 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 846/1696] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 429 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:38:03.688934 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496f92f-a8dd-4be3-bce1-ea66fde50e00 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:03.689226 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496f92f-a8dd-4be3-bce1-ea66fde50e00 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:03.690380 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496f92f-a8dd-4be3-bce1-ea66fde50e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:03.737509 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-4496f92f-a8dd-4be3-bce1-ea66fde50e00 None None] MFA Rules not processed for user `f26a832b69564f2a8387215afba6730e`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:03.739658 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-4496f92f-a8dd-4be3-bce1-ea66fde50e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:03.792328 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e934e4-34ed-4a05-87dc-d786b09e4ffb admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:03.799827 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b84471-752e-4b81-b3b6-003bc47baad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:03.802617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b84471-752e-4b81-b3b6-003bc47baad7 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:03.803444 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b84471-752e-4b81-b3b6-003bc47baad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:03.803941 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b84471-752e-4b81-b3b6-003bc47baad7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:03.804903 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b84471-752e-4b81-b3b6-003bc47baad7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:03.807518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b84471-752e-4b81-b3b6-003bc47baad7 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:03.830380 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 851/1697] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:03.840041 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b84471-752e-4b81-b3b6-003bc47baad7 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:03.860542 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b84471-752e-4b81-b3b6-003bc47baad7 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:03.879566 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa263847-3f4d-425e-9237-796655ca702c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:03.880606 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa263847-3f4d-425e-9237-796655ca702c admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:03.880891 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa263847-3f4d-425e-9237-796655ca702c admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:03.881169 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa263847-3f4d-425e-9237-796655ca702c admin admin] PATH_INFO: `/v3/projects/3b6fdee1681f41e388d25b044414b360/users/c135e6837b7b4d02963e5ae2e81c1aff/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:03.887161 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa263847-3f4d-425e-9237-796655ca702c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b6fdee1681f41e388d25b044414b360, user_id=c135e6837b7b4d02963e5ae2e81c1aff, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:03.892165 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 847/1698] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:03.895373 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa263847-3f4d-425e-9237-796655ca702c admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:03.907258 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4496f92f-a8dd-4be3-bce1-ea66fde50e00 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:03.909881 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-4496f92f-a8dd-4be3-bce1-ea66fde50e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:03.929005 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 852/1699] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:03 2022] PUT /identity/v3/projects/3b6fdee1681f41e388d25b044414b360/users/c135e6837b7b4d02963e5ae2e81c1aff/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:03.940523 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f9b05-9277-4992-94ff-a8611153b392 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:03.941108 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f9b05-9277-4992-94ff-a8611153b392 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:03.941642 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f9b05-9277-4992-94ff-a8611153b392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:04.006185 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-ae6f9b05-9277-4992-94ff-a8611153b392 None None] MFA Rules not processed for user `c135e6837b7b4d02963e5ae2e81c1aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:04.011966 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ae6f9b05-9277-4992-94ff-a8611153b392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:04.110431 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:04.133799 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4496f92f-a8dd-4be3-bce1-ea66fde50e00 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:04.136338 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:04.225038 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 853/1700] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:03 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:38:04.238178 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f9b05-9277-4992-94ff-a8611153b392 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:04.242264 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ae6f9b05-9277-4992-94ff-a8611153b392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:04.335251 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ada12e-18e5-43c8-a2d2-d0810626a7e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:04.343547 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ada12e-18e5-43c8-a2d2-d0810626a7e1 service nova] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:04.343917 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ada12e-18e5-43c8-a2d2-d0810626a7e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:04.344272 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ada12e-18e5-43c8-a2d2-d0810626a7e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:04.400634 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f9b05-9277-4992-94ff-a8611153b392 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:04.409667 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ecdeaf-64a9-4388-b4aa-e8c019808dbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:04.410513 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ecdeaf-64a9-4388-b4aa-e8c019808dbe service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:04.411071 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ecdeaf-64a9-4388-b4aa-e8c019808dbe service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:04.413164 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ecdeaf-64a9-4388-b4aa-e8c019808dbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:04.438103 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ecdeaf-64a9-4388-b4aa-e8c019808dbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:04.440016 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ecdeaf-64a9-4388-b4aa-e8c019808dbe service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:04.471970 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 854/1701] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:04 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:38:04.477892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9ba43-22cf-418a-86e9-dbcd42693925 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:04.480651 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9ba43-22cf-418a-86e9-dbcd42693925 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:04.482869 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9ba43-22cf-418a-86e9-dbcd42693925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:04.499797 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ada12e-18e5-43c8-a2d2-d0810626a7e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:04.503336 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ada12e-18e5-43c8-a2d2-d0810626a7e1 service nova] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:04.573030 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 848/1702] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 666 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:38:04.578195 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c4c9ba43-22cf-418a-86e9-dbcd42693925 None None] MFA Rules not processed for user `c815423179f44b219e4f8572b10355c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:04.580117 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c4c9ba43-22cf-418a-86e9-dbcd42693925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:04.590388 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ada12e-18e5-43c8-a2d2-d0810626a7e1 service nova] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:04.623608 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ada12e-18e5-43c8-a2d2-d0810626a7e1 service nova] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:04.655275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508fd61-a780-4508-9858-30da929631c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:04.657575 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508fd61-a780-4508-9858-30da929631c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:04.660714 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508fd61-a780-4508-9858-30da929631c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:04.661844 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508fd61-a780-4508-9858-30da929631c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:04.670951 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-f508fd61-a780-4508-9858-30da929631c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:04.891299 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 855/1703] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 417 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:04.915685 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f508fd61-a780-4508-9858-30da929631c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:04.917153 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f508fd61-a780-4508-9858-30da929631c2 service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:04.950184 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 849/1704] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:04 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:38:06.536523 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95799ef5-5186-40cb-918d-473acb09aa6e None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:06.538465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95799ef5-5186-40cb-918d-473acb09aa6e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:06.540037 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95799ef5-5186-40cb-918d-473acb09aa6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:06.683944 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-95799ef5-5186-40cb-918d-473acb09aa6e None None] MFA Rules not processed for user `5c63e0b6ceed4f7cb2e6f9269eb85795`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:06.685761 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-95799ef5-5186-40cb-918d-473acb09aa6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:06.969706 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 856/1705] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Jun 15 19:38:06 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 427 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:07.054499 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508fd61-a780-4508-9858-30da929631c2 service cinder] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:07.062313 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-f508fd61-a780-4508-9858-30da929631c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:07.139626 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508fd61-a780-4508-9858-30da929631c2 service cinder] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:07.166346 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d931a1-dc25-4834-b110-7ced6a7ec62a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:07.170917 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d931a1-dc25-4834-b110-7ced6a7ec62a service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:07.170917 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d931a1-dc25-4834-b110-7ced6a7ec62a service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:07.170917 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d931a1-dc25-4834-b110-7ced6a7ec62a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:07.241032 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d931a1-dc25-4834-b110-7ced6a7ec62a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:07.249911 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d931a1-dc25-4834-b110-7ced6a7ec62a service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:07.275507 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 850/1706] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:38:07.579922 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95799ef5-5186-40cb-918d-473acb09aa6e None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:07.587056 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-95799ef5-5186-40cb-918d-473acb09aa6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:07.786443 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:07.903584 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95799ef5-5186-40cb-918d-473acb09aa6e None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:07.929426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc0b1b1-f553-418c-a89f-7414c17449d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:07.930957 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc0b1b1-f553-418c-a89f-7414c17449d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:07.935275 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc0b1b1-f553-418c-a89f-7414c17449d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:07.949513 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc0b1b1-f553-418c-a89f-7414c17449d1 admin admin] PATH_INFO: `/v3/projects/12496d3308984f88a079e8ce8596d03a` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:07.956520 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc0b1b1-f553-418c-a89f-7414c17449d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12496d3308984f88a079e8ce8596d03a)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:07.965994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc0b1b1-f553-418c-a89f-7414c17449d1 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:08.184401 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1bc0b1b1-f553-418c-a89f-7414c17449d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12496d3308984f88a079e8ce8596d03a', 'request_id': 'req-1bc0b1b1-f553-418c-a89f-7414c17449d1'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:08.208474 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1bc0b1b1-f553-418c-a89f-7414c17449d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12496d3308984f88a079e8ce8596d03a', 'request_id': 'req-1bc0b1b1-f553-418c-a89f-7414c17449d1'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:08.246874 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1bc0b1b1-f553-418c-a89f-7414c17449d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12496d3308984f88a079e8ce8596d03a', 'request_id': 'req-1bc0b1b1-f553-418c-a89f-7414c17449d1'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:08.349314 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1bc0b1b1-f553-418c-a89f-7414c17449d1 admin admin] The token cache is being invalidate because project 12496d3308984f88a079e8ce8596d03a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:08.349842 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1bc0b1b1-f553-418c-a89f-7414c17449d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:08.363236 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1bc0b1b1-f553-418c-a89f-7414c17449d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:08.379362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 857/1707] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:07 2022] DELETE /identity/v3/projects/12496d3308984f88a079e8ce8596d03a => generated 0 bytes in 801 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:08.385754 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9878748-45ea-4f1a-ade0-62eeed018461 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:08.386759 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9878748-45ea-4f1a-ade0-62eeed018461 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:08.387265 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9878748-45ea-4f1a-ade0-62eeed018461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:08.444791 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-b9878748-45ea-4f1a-ade0-62eeed018461 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:08.446455 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-b9878748-45ea-4f1a-ade0-62eeed018461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:08.493000 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:08.617916 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 851/1708] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:08.627524 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee3874-e1b8-4704-8be7-63fb1832b2d5 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:08.628062 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee3874-e1b8-4704-8be7-63fb1832b2d5 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:08.628159 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee3874-e1b8-4704-8be7-63fb1832b2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:08.660689 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-b8ee3874-e1b8-4704-8be7-63fb1832b2d5 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:08.663394 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b8ee3874-e1b8-4704-8be7-63fb1832b2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:08.685363 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 858/1709] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:08.685997 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9878748-45ea-4f1a-ade0-62eeed018461 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:08.695878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9878748-45ea-4f1a-ade0-62eeed018461 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:08.719940 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87a318-1938-4e82-b192-0eea1d5bfb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:08.720969 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87a318-1938-4e82-b192-0eea1d5bfb9f admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:08.720969 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87a318-1938-4e82-b192-0eea1d5bfb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:08.721374 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87a318-1938-4e82-b192-0eea1d5bfb9f admin admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:08.722228 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c87a318-1938-4e82-b192-0eea1d5bfb9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:38:08.722602 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c87a318-1938-4e82-b192-0eea1d5bfb9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:08.726581 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c87a318-1938-4e82-b192-0eea1d5bfb9f admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:08.783610 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 852/1710] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:38:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:38:08.790149 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ee3874-e1b8-4704-8be7-63fb1832b2d5 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:08.813935 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ee3874-e1b8-4704-8be7-63fb1832b2d5 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:08.821212 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd56e8-e8af-4976-8050-dea2213b3223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:08.823034 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd56e8-e8af-4976-8050-dea2213b3223 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:08.823797 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd56e8-e8af-4976-8050-dea2213b3223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:08.824391 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd56e8-e8af-4976-8050-dea2213b3223 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:08.826036 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd56e8-e8af-4976-8050-dea2213b3223 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:08.828334 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd56e8-e8af-4976-8050-dea2213b3223 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:08.879874 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 859/1711] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:08 2022] POST /identity/v3/projects => generated 381 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:08.880837 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87a318-1938-4e82-b192-0eea1d5bfb9f admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:08.901653 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87a318-1938-4e82-b192-0eea1d5bfb9f admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:09.078763 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d64f15-31eb-40c6-b1fe-31cc5bf2577d None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:09.079170 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d64f15-31eb-40c6-b1fe-31cc5bf2577d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:09.079458 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d64f15-31eb-40c6-b1fe-31cc5bf2577d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:09.217011 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e2f0c1-dc45-4a86-952c-b94b38b0893b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:09.217839 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e2f0c1-dc45-4a86-952c-b94b38b0893b admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:09.218425 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e2f0c1-dc45-4a86-952c-b94b38b0893b admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:09.218871 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e2f0c1-dc45-4a86-952c-b94b38b0893b admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:09.219669 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e2f0c1-dc45-4a86-952c-b94b38b0893b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:09.221966 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e2f0c1-dc45-4a86-952c-b94b38b0893b admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:09.223408 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-70d64f15-31eb-40c6-b1fe-31cc5bf2577d None None] MFA Rules not processed for user `e1635576dfe242f88ec1abb7be6359fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:09.231671 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-70d64f15-31eb-40c6-b1fe-31cc5bf2577d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:09.372695 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 853/1712] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:08 2022] POST /identity/v3/users => generated 291 bytes in 492 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:09.374055 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e2f0c1-dc45-4a86-952c-b94b38b0893b admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:09.428003 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e2f0c1-dc45-4a86-952c-b94b38b0893b admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:09.453266 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74b445-0542-4ce6-93ae-d2e03da76ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:09.454844 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74b445-0542-4ce6-93ae-d2e03da76ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:09.464515 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74b445-0542-4ce6-93ae-d2e03da76ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:09.465161 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74b445-0542-4ce6-93ae-d2e03da76ae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:09.466650 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f74b445-0542-4ce6-93ae-d2e03da76ae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:09.481368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f74b445-0542-4ce6-93ae-d2e03da76ae3 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:09.541197 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 854/1713] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:09.542519 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74b445-0542-4ce6-93ae-d2e03da76ae3 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:09.599142 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74b445-0542-4ce6-93ae-d2e03da76ae3 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:09.620897 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c113a-46bd-4296-b659-a99037b1399c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:09.624234 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c113a-46bd-4296-b659-a99037b1399c admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:09.627538 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c113a-46bd-4296-b659-a99037b1399c admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:09.628008 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c113a-46bd-4296-b659-a99037b1399c admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:09.628862 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c113a-46bd-4296-b659-a99037b1399c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:09.632679 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c113a-46bd-4296-b659-a99037b1399c admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:09.648198 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 860/1714] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 563 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:09.700048 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 855/1715] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:09.704275 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d64f15-31eb-40c6-b1fe-31cc5bf2577d None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:09.744238 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d64f15-31eb-40c6-b1fe-31cc5bf2577d None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:09.762384 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c281c18f-ca42-4549-9d7f-c2abd6175714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:09.763427 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c281c18f-ca42-4549-9d7f-c2abd6175714 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:09.763712 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c281c18f-ca42-4549-9d7f-c2abd6175714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:09.763991 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c281c18f-ca42-4549-9d7f-c2abd6175714 admin admin] PATH_INFO: `/v3/projects/ba45712ef62c42dbb408d059dac757c6/users/3e24302094fc48e79c150c1b641a39af/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:09.825770 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c281c18f-ca42-4549-9d7f-c2abd6175714 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba45712ef62c42dbb408d059dac757c6, user_id=3e24302094fc48e79c150c1b641a39af, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:09.829259 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c281c18f-ca42-4549-9d7f-c2abd6175714 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:09.860611 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:09.882428 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 861/1716] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:09 2022] PUT /identity/v3/projects/ba45712ef62c42dbb408d059dac757c6/users/3e24302094fc48e79c150c1b641a39af/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:09.894275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3419fa-68da-45ce-9834-5ed86d847711 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:09.894958 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3419fa-68da-45ce-9834-5ed86d847711 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:09.895381 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3419fa-68da-45ce-9834-5ed86d847711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:09.925788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684d516-9358-4946-a952-84984c796acc None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:09.926506 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684d516-9358-4946-a952-84984c796acc None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:09.926932 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684d516-9358-4946-a952-84984c796acc None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:09.928871 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 862/1717] 10.222.0.15 () {54 vars in 921 bytes} [Wed Jun 15 19:38:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:38:09.981277 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-ad3419fa-68da-45ce-9834-5ed86d847711 None None] MFA Rules not processed for user `3e24302094fc48e79c150c1b641a39af`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:09.984407 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-ad3419fa-68da-45ce-9834-5ed86d847711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:10.016488 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5cb451-d202-409a-8b36-0427cfe62612 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:10.017014 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5cb451-d202-409a-8b36-0427cfe62612 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:10.017485 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5cb451-d202-409a-8b36-0427cfe62612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:10.076809 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-3b5cb451-d202-409a-8b36-0427cfe62612 None None] MFA Rules not processed for user `3e74ac44e5814122a3226ac023f3688a`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:10.087007 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3b5cb451-d202-409a-8b36-0427cfe62612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:10.277645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 856/1718] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:10.278794 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3419fa-68da-45ce-9834-5ed86d847711 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:10.283003 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-ad3419fa-68da-45ce-9834-5ed86d847711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:10.323029 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:10.410718 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 863/1719] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Jun 15 19:38:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:38:10.411910 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5cb451-d202-409a-8b36-0427cfe62612 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:10.424893 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3b5cb451-d202-409a-8b36-0427cfe62612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:10.672018 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3419fa-68da-45ce-9834-5ed86d847711 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:10.699321 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b347f4-4d05-4d88-b938-2bf1fba46d1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:10.704647 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b347f4-4d05-4d88-b938-2bf1fba46d1c service nova] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:10.706956 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b347f4-4d05-4d88-b938-2bf1fba46d1c service nova] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:10.709656 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b347f4-4d05-4d88-b938-2bf1fba46d1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:10.734342 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5cb451-d202-409a-8b36-0427cfe62612 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:10.751902 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9786a7c-c328-4d3f-b5ec-edf5923864ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:10.752502 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9786a7c-c328-4d3f-b5ec-edf5923864ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:10.752785 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9786a7c-c328-4d3f-b5ec-edf5923864ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:10.753126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9786a7c-c328-4d3f-b5ec-edf5923864ad admin admin] PATH_INFO: `/v3/users/9c6fd3dfbd5a485c9b3c6091bf18667d` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:10.759019 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9786a7c-c328-4d3f-b5ec-edf5923864ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c6fd3dfbd5a485c9b3c6091bf18667d)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:10.776856 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9786a7c-c328-4d3f-b5ec-edf5923864ad admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:10.976002 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b347f4-4d05-4d88-b938-2bf1fba46d1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:10.978468 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b347f4-4d05-4d88-b938-2bf1fba46d1c service nova] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:10.979145 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f9786a7c-c328-4d3f-b5ec-edf5923864ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c6fd3dfbd5a485c9b3c6091bf18667d'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:10.980049 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f9786a7c-c328-4d3f-b5ec-edf5923864ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c6fd3dfbd5a485c9b3c6091bf18667d'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:10.980843 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f9786a7c-c328-4d3f-b5ec-edf5923864ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c6fd3dfbd5a485c9b3c6091bf18667d'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:11.027315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 857/1720] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 752 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:38:11.028370 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f9786a7c-c328-4d3f-b5ec-edf5923864ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c6fd3dfbd5a485c9b3c6091bf18667d'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:11.029689 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b347f4-4d05-4d88-b938-2bf1fba46d1c service nova] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:11.044665 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f9786a7c-c328-4d3f-b5ec-edf5923864ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c6fd3dfbd5a485c9b3c6091bf18667d'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:11.047306 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b347f4-4d05-4d88-b938-2bf1fba46d1c service nova] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:11.060468 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab406a-f130-4013-b57f-bb2da35c894c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:11.061130 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab406a-f130-4013-b57f-bb2da35c894c admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:11.061418 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab406a-f130-4013-b57f-bb2da35c894c admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:11.061762 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab406a-f130-4013-b57f-bb2da35c894c admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:11.062619 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab406a-f130-4013-b57f-bb2da35c894c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:11.064731 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab406a-f130-4013-b57f-bb2da35c894c admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:11.080002 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 864/1721] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:38:10 2022] DELETE /identity/v3/users/9c6fd3dfbd5a485c9b3c6091bf18667d => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:11.081117 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9786a7c-c328-4d3f-b5ec-edf5923864ad admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:11.097479 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f9786a7c-c328-4d3f-b5ec-edf5923864ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:11.159700 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:11.162629 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 858/1722] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:11 2022] POST /identity/v3/projects => generated 381 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:11.181138 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56f774-38d4-4067-8947-f6b522e76743 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:11.182455 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56f774-38d4-4067-8947-f6b522e76743 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:11.182767 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56f774-38d4-4067-8947-f6b522e76743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:11.233936 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-7c56f774-38d4-4067-8947-f6b522e76743 None None] MFA Rules not processed for user `f26a832b69564f2a8387215afba6730e`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:11.235654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-7c56f774-38d4-4067-8947-f6b522e76743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:11.347870 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9786a7c-c328-4d3f-b5ec-edf5923864ad admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:11.494428 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 859/1723] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:11.495766 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3068e3-be96-4d94-b2af-84fdb9dee0af None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:11.496056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3068e3-be96-4d94-b2af-84fdb9dee0af None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:11.496330 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3068e3-be96-4d94-b2af-84fdb9dee0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:11.551767 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-1d3068e3-be96-4d94-b2af-84fdb9dee0af None None] MFA Rules not processed for user `b2103a7ed6824444a60fa5573e7ad39e`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:11.554419 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-1d3068e3-be96-4d94-b2af-84fdb9dee0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:11.619117 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:11.621648 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b0dea-b109-4b84-bf10-9ca5ccfee742 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:11.623245 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b0dea-b109-4b84-bf10-9ca5ccfee742 service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:11.624953 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b0dea-b109-4b84-bf10-9ca5ccfee742 service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:11.631027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b0dea-b109-4b84-bf10-9ca5ccfee742 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:11.636944 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-539b0dea-b109-4b84-bf10-9ca5ccfee742 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:11.695408 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b0dea-b109-4b84-bf10-9ca5ccfee742 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:11.698358 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b0dea-b109-4b84-bf10-9ca5ccfee742 service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:11.699289 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 860/1724] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:11.705656 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724cc14-34e8-40ff-8c01-b0ca8c20050f None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:11.705656 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724cc14-34e8-40ff-8c01-b0ca8c20050f None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:11.705656 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724cc14-34e8-40ff-8c01-b0ca8c20050f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:11.739270 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 865/1725] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 659 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:38:11.740908 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b0dea-b109-4b84-bf10-9ca5ccfee742 service nova] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:11.746243 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-3724cc14-34e8-40ff-8c01-b0ca8c20050f None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:11.747357 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-539b0dea-b109-4b84-bf10-9ca5ccfee742 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:11.747592 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-3724cc14-34e8-40ff-8c01-b0ca8c20050f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:11.925302 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 861/1726] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:11.925302 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3724cc14-34e8-40ff-8c01-b0ca8c20050f None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:11.925969 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-3724cc14-34e8-40ff-8c01-b0ca8c20050f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:11.948613 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3724cc14-34e8-40ff-8c01-b0ca8c20050f None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:11.948803 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b0dea-b109-4b84-bf10-9ca5ccfee742 service nova] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:11.962819 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b774e0d-dfbf-4ab1-af04-a3b18ebc946c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:11.963113 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ddd74-0b0b-4395-b24f-eb99cd66f56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:11.963739 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b774e0d-dfbf-4ab1-af04-a3b18ebc946c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:11.964063 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b774e0d-dfbf-4ab1-af04-a3b18ebc946c admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:11.964341 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b774e0d-dfbf-4ab1-af04-a3b18ebc946c admin admin] PATH_INFO: `/v3/users/c1786f9a1ccb4eff8a8deab64dd6e1dd` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:11.964655 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ddd74-0b0b-4395-b24f-eb99cd66f56d admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:11.965196 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ddd74-0b0b-4395-b24f-eb99cd66f56d admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:11.965799 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ddd74-0b0b-4395-b24f-eb99cd66f56d admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:11.967891 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0ddd74-0b0b-4395-b24f-eb99cd66f56d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:11.973883 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0ddd74-0b0b-4395-b24f-eb99cd66f56d admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:11.980487 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b774e0d-dfbf-4ab1-af04-a3b18ebc946c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1786f9a1ccb4eff8a8deab64dd6e1dd)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:11.991254 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b774e0d-dfbf-4ab1-af04-a3b18ebc946c admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:12.022444 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 866/1727] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:11 2022] POST /identity/v3/users => generated 297 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:12.023491 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ddd74-0b0b-4395-b24f-eb99cd66f56d admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:12.039086 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3a0ddd74-0b0b-4395-b24f-eb99cd66f56d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:12.168246 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:12.170504 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-1b774e0d-dfbf-4ab1-af04-a3b18ebc946c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1786f9a1ccb4eff8a8deab64dd6e1dd'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:12.174540 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-1b774e0d-dfbf-4ab1-af04-a3b18ebc946c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1786f9a1ccb4eff8a8deab64dd6e1dd'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:12.191521 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-1b774e0d-dfbf-4ab1-af04-a3b18ebc946c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1786f9a1ccb4eff8a8deab64dd6e1dd'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:12.192436 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-1b774e0d-dfbf-4ab1-af04-a3b18ebc946c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1786f9a1ccb4eff8a8deab64dd6e1dd'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:12.201806 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-1b774e0d-dfbf-4ab1-af04-a3b18ebc946c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1786f9a1ccb4eff8a8deab64dd6e1dd'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:12.229934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ddd74-0b0b-4395-b24f-eb99cd66f56d admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:12.238837 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e635cfa6-916e-4dc3-bd9c-890107457851 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:12.239514 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635cfa6-916e-4dc3-bd9c-890107457851 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:12.239790 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 862/1728] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:38:11 2022] DELETE /identity/v3/users/c1786f9a1ccb4eff8a8deab64dd6e1dd => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:12.240663 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b774e0d-dfbf-4ab1-af04-a3b18ebc946c admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:12.242795 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635cfa6-916e-4dc3-bd9c-890107457851 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:12.243182 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635cfa6-916e-4dc3-bd9c-890107457851 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:12.245673 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e635cfa6-916e-4dc3-bd9c-890107457851 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:12.246867 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-1b774e0d-dfbf-4ab1-af04-a3b18ebc946c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:12.260282 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e635cfa6-916e-4dc3-bd9c-890107457851 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:12.261856 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e635cfa6-916e-4dc3-bd9c-890107457851 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:12.280115 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 867/1729] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:38:12.284329 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c700084-8c47-4c4a-81fe-88c2dd6b4a42 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:12.284539 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c700084-8c47-4c4a-81fe-88c2dd6b4a42 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:12.299204 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c700084-8c47-4c4a-81fe-88c2dd6b4a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:12.371636 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-8c700084-8c47-4c4a-81fe-88c2dd6b4a42 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:12.377698 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8c700084-8c47-4c4a-81fe-88c2dd6b4a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:12.380263 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:12.440415 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b774e0d-dfbf-4ab1-af04-a3b18ebc946c admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:12.446226 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04da7c-fbc5-4009-b274-2bb63de2a0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:12.448975 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04da7c-fbc5-4009-b274-2bb63de2a0e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:12.449267 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04da7c-fbc5-4009-b274-2bb63de2a0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:12.451092 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04da7c-fbc5-4009-b274-2bb63de2a0e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:12.451724 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f04da7c-fbc5-4009-b274-2bb63de2a0e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:12.461737 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f04da7c-fbc5-4009-b274-2bb63de2a0e2 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:12.482067 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 863/1730] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:12.487119 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04da7c-fbc5-4009-b274-2bb63de2a0e2 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:12.513814 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04da7c-fbc5-4009-b274-2bb63de2a0e2 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:12.513992 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 868/1731] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:12.525188 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c700084-8c47-4c4a-81fe-88c2dd6b4a42 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:12.526865 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8c700084-8c47-4c4a-81fe-88c2dd6b4a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:12.528125 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b36c6-2c6b-4025-8da2-a2ed42af0a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:12.529687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b36c6-2c6b-4025-8da2-a2ed42af0a88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:12.530054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b36c6-2c6b-4025-8da2-a2ed42af0a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:12.530458 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b36c6-2c6b-4025-8da2-a2ed42af0a88 admin admin] PATH_INFO: `/v3/projects/77922c50e37b48d4907d81074515a392/users/2878e3b2e93044ddb32cc40347ff9b2c/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:12.553821 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2b36c6-2c6b-4025-8da2-a2ed42af0a88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77922c50e37b48d4907d81074515a392, user_id=2878e3b2e93044ddb32cc40347ff9b2c, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:12.556924 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2b36c6-2c6b-4025-8da2-a2ed42af0a88 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:12.577177 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 864/1732] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:12 2022] PUT /identity/v3/projects/77922c50e37b48d4907d81074515a392/users/2878e3b2e93044ddb32cc40347ff9b2c/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:12.581072 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b36c6-2c6b-4025-8da2-a2ed42af0a88 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:12.649617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c700084-8c47-4c4a-81fe-88c2dd6b4a42 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:12.701567 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b36c6-2c6b-4025-8da2-a2ed42af0a88 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:12.712568 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea205ec-018e-4f9c-ae06-68a76cec6bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:12.713275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea205ec-018e-4f9c-ae06-68a76cec6bda admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:12.713558 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea205ec-018e-4f9c-ae06-68a76cec6bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:12.713836 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea205ec-018e-4f9c-ae06-68a76cec6bda admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:12.714683 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea205ec-018e-4f9c-ae06-68a76cec6bda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:12.717714 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea205ec-018e-4f9c-ae06-68a76cec6bda admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:12.740952 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 865/1733] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:12.744936 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea205ec-018e-4f9c-ae06-68a76cec6bda admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:12.765502 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea205ec-018e-4f9c-ae06-68a76cec6bda admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:12.767606 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de76bd6-6956-47bc-9f59-8543739f3ca4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:12.771106 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de76bd6-6956-47bc-9f59-8543739f3ca4 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:12.771396 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de76bd6-6956-47bc-9f59-8543739f3ca4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:12.772196 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de76bd6-6956-47bc-9f59-8543739f3ca4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:12.779354 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928428ee-be66-45f2-9011-8a2bc231e0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:12.780840 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928428ee-be66-45f2-9011-8a2bc231e0b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:12.781129 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928428ee-be66-45f2-9011-8a2bc231e0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:12.781536 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928428ee-be66-45f2-9011-8a2bc231e0b7 admin admin] PATH_INFO: `/v3/projects/77922c50e37b48d4907d81074515a392/users/2878e3b2e93044ddb32cc40347ff9b2c/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:12.789137 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928428ee-be66-45f2-9011-8a2bc231e0b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77922c50e37b48d4907d81074515a392, user_id=2878e3b2e93044ddb32cc40347ff9b2c, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:12.793045 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928428ee-be66-45f2-9011-8a2bc231e0b7 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:12.800733 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de76bd6-6956-47bc-9f59-8543739f3ca4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:12.805376 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de76bd6-6956-47bc-9f59-8543739f3ca4 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:12.823494 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 869/1734] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:38:12.823942 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 866/1735] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:12 2022] PUT /identity/v3/projects/77922c50e37b48d4907d81074515a392/users/2878e3b2e93044ddb32cc40347ff9b2c/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:12.835929 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beae550-e3ee-48cb-b7a2-5bb5a824955c None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:12.836550 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beae550-e3ee-48cb-b7a2-5bb5a824955c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:12.836679 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beae550-e3ee-48cb-b7a2-5bb5a824955c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:12.880186 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-0beae550-e3ee-48cb-b7a2-5bb5a824955c None None] MFA Rules not processed for user `2878e3b2e93044ddb32cc40347ff9b2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:12.881681 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0beae550-e3ee-48cb-b7a2-5bb5a824955c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:12.905053 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:13.008340 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 870/1736] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:12 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:13.016424 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928428ee-be66-45f2-9011-8a2bc231e0b7 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:13.018643 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-928428ee-be66-45f2-9011-8a2bc231e0b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:13.174434 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928428ee-be66-45f2-9011-8a2bc231e0b7 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:13.184865 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c311e0-766c-4545-9d8e-01ff6bc9c3f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:13.186118 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c311e0-766c-4545-9d8e-01ff6bc9c3f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:13.189315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c311e0-766c-4545-9d8e-01ff6bc9c3f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:13.189628 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c311e0-766c-4545-9d8e-01ff6bc9c3f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:13.218650 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c311e0-766c-4545-9d8e-01ff6bc9c3f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:13.228816 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c311e0-766c-4545-9d8e-01ff6bc9c3f8 service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:13.282390 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 867/1737] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:13 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:38:13.958747 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beae550-e3ee-48cb-b7a2-5bb5a824955c None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:13.973724 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beae550-e3ee-48cb-b7a2-5bb5a824955c None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:13.984802 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbcfff0-2f44-403e-81b4-9c9cf82c7ffa tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2878e3b2e93044ddb32cc40347ff9b2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77922c50e37b48d4907d81074515a392', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:13.987815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbcfff0-2f44-403e-81b4-9c9cf82c7ffa tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:13.989938 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbcfff0-2f44-403e-81b4-9c9cf82c7ffa tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:13.989938 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbcfff0-2f44-403e-81b4-9c9cf82c7ffa tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:13.991113 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbcfff0-2f44-403e-81b4-9c9cf82c7ffa tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:38:13.991113 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbcfff0-2f44-403e-81b4-9c9cf82c7ffa tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:13.994845 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbcfff0-2f44-403e-81b4-9c9cf82c7ffa tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:14.023997 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 871/1738] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:38:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:38:14.028769 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c311e0-766c-4545-9d8e-01ff6bc9c3f8 service cinder] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:14.044106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c311e0-766c-4545-9d8e-01ff6bc9c3f8 service cinder] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:14.054365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de39bf63-f04f-4974-8a5b-a24b6b7ad16d tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2878e3b2e93044ddb32cc40347ff9b2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77922c50e37b48d4907d81074515a392', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:14.055132 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39bf63-f04f-4974-8a5b-a24b6b7ad16d tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:14.055509 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39bf63-f04f-4974-8a5b-a24b6b7ad16d tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:14.055881 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39bf63-f04f-4974-8a5b-a24b6b7ad16d tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:14.056706 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de39bf63-f04f-4974-8a5b-a24b6b7ad16d tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:14.059222 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de39bf63-f04f-4974-8a5b-a24b6b7ad16d tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:14.099396 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 868/1739] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:14 2022] POST /identity/v3/projects => generated 354 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:14.101165 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbcfff0-2f44-403e-81b4-9c9cf82c7ffa tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:14.115577 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbcfff0-2f44-403e-81b4-9c9cf82c7ffa tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:14.239500 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58f39d4-1e7e-4c4a-8930-b5fc805b0fdc tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2878e3b2e93044ddb32cc40347ff9b2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77922c50e37b48d4907d81074515a392', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:14.241067 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f39d4-1e7e-4c4a-8930-b5fc805b0fdc tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:14.241706 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f39d4-1e7e-4c4a-8930-b5fc805b0fdc tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:14.241706 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f39d4-1e7e-4c4a-8930-b5fc805b0fdc tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:14.243305 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58f39d4-1e7e-4c4a-8930-b5fc805b0fdc tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:38:14.243625 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58f39d4-1e7e-4c4a-8930-b5fc805b0fdc tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:14.245279 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58f39d4-1e7e-4c4a-8930-b5fc805b0fdc tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:14.277645 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 872/1740] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:38:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:38:14.341820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de39bf63-f04f-4974-8a5b-a24b6b7ad16d tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:14.358522 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de39bf63-f04f-4974-8a5b-a24b6b7ad16d tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:14.376861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853bdd92-ad9d-439c-99f9-e0c00221e58f tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2878e3b2e93044ddb32cc40347ff9b2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77922c50e37b48d4907d81074515a392', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:14.377769 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bdd92-ad9d-439c-99f9-e0c00221e58f tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:14.378123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bdd92-ad9d-439c-99f9-e0c00221e58f tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:14.378675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bdd92-ad9d-439c-99f9-e0c00221e58f tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] PATH_INFO: `/v3/projects/2919e96239ca41e7965c97ce6f04d36a` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:14.379708 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853bdd92-ad9d-439c-99f9-e0c00221e58f tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2919e96239ca41e7965c97ce6f04d36a)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:14.381813 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853bdd92-ad9d-439c-99f9-e0c00221e58f tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:14.430361 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:14.494831 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58f39d4-1e7e-4c4a-8930-b5fc805b0fdc tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:14.498188 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f58f39d4-1e7e-4c4a-8930-b5fc805b0fdc tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:14.521286 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-853bdd92-ad9d-439c-99f9-e0c00221e58f tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2919e96239ca41e7965c97ce6f04d36a', 'request_id': 'req-853bdd92-ad9d-439c-99f9-e0c00221e58f'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:14.535662 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-853bdd92-ad9d-439c-99f9-e0c00221e58f tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2919e96239ca41e7965c97ce6f04d36a', 'request_id': 'req-853bdd92-ad9d-439c-99f9-e0c00221e58f'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:14.561101 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-853bdd92-ad9d-439c-99f9-e0c00221e58f tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2919e96239ca41e7965c97ce6f04d36a', 'request_id': 'req-853bdd92-ad9d-439c-99f9-e0c00221e58f'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:14.596976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-853bdd92-ad9d-439c-99f9-e0c00221e58f tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] The token cache is being invalidate because project 2919e96239ca41e7965c97ce6f04d36a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:14.597372 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-853bdd92-ad9d-439c-99f9-e0c00221e58f tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:14.599819 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-853bdd92-ad9d-439c-99f9-e0c00221e58f tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:14.603885 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 869/1741] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:14 2022] DELETE /identity/v3/projects/2919e96239ca41e7965c97ce6f04d36a => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:14.710866 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58f39d4-1e7e-4c4a-8930-b5fc805b0fdc tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:14.763261 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:14.795805 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dccc986-c49b-491b-aaf4-2e5794579e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:14.797082 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dccc986-c49b-491b-aaf4-2e5794579e95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:14.797427 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dccc986-c49b-491b-aaf4-2e5794579e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:14.797713 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dccc986-c49b-491b-aaf4-2e5794579e95 admin admin] PATH_INFO: `/v3/projects/2305125818b54552bc09b0942be3e4d5` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:14.799260 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dccc986-c49b-491b-aaf4-2e5794579e95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2305125818b54552bc09b0942be3e4d5)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:14.801327 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dccc986-c49b-491b-aaf4-2e5794579e95 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:14.965698 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853bdd92-ad9d-439c-99f9-e0c00221e58f tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:14.967778 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-853bdd92-ad9d-439c-99f9-e0c00221e58f tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:14.999192 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6dccc986-c49b-491b-aaf4-2e5794579e95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2305125818b54552bc09b0942be3e4d5', 'request_id': 'req-6dccc986-c49b-491b-aaf4-2e5794579e95'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:15.013142 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853bdd92-ad9d-439c-99f9-e0c00221e58f tempest-VolumeQuotaClassesTest-1555211958 tempest-VolumeQuotaClassesTest-1555211958-project-admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:15.029104 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6dccc986-c49b-491b-aaf4-2e5794579e95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2305125818b54552bc09b0942be3e4d5', 'request_id': 'req-6dccc986-c49b-491b-aaf4-2e5794579e95'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:15.051746 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6dccc986-c49b-491b-aaf4-2e5794579e95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2305125818b54552bc09b0942be3e4d5', 'request_id': 'req-6dccc986-c49b-491b-aaf4-2e5794579e95'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:15.101192 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6dccc986-c49b-491b-aaf4-2e5794579e95 admin admin] The token cache is being invalidate because project 2305125818b54552bc09b0942be3e4d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:15.101662 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6dccc986-c49b-491b-aaf4-2e5794579e95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:15.104643 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6dccc986-c49b-491b-aaf4-2e5794579e95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:15.112618 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 873/1742] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:14 2022] DELETE /identity/v3/projects/2305125818b54552bc09b0942be3e4d5 => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:15.113540 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dccc986-c49b-491b-aaf4-2e5794579e95 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:15.116656 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6dccc986-c49b-491b-aaf4-2e5794579e95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:15.257772 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9ac9d-797c-41a0-a2c2-bf75f1424caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:15.258693 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9ac9d-797c-41a0-a2c2-bf75f1424caf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:15.258979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9ac9d-797c-41a0-a2c2-bf75f1424caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:15.259259 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9ac9d-797c-41a0-a2c2-bf75f1424caf admin admin] PATH_INFO: `/v3/projects/6f098b8fc390430b9f98879e6afcd1b7` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:15.266037 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e9ac9d-797c-41a0-a2c2-bf75f1424caf admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f098b8fc390430b9f98879e6afcd1b7)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:15.268045 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e9ac9d-797c-41a0-a2c2-bf75f1424caf admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:15.372898 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dccc986-c49b-491b-aaf4-2e5794579e95 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:15.377419 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-88e9ac9d-797c-41a0-a2c2-bf75f1424caf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f098b8fc390430b9f98879e6afcd1b7', 'request_id': 'req-88e9ac9d-797c-41a0-a2c2-bf75f1424caf'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:15.396834 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-88e9ac9d-797c-41a0-a2c2-bf75f1424caf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f098b8fc390430b9f98879e6afcd1b7', 'request_id': 'req-88e9ac9d-797c-41a0-a2c2-bf75f1424caf'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:15.409729 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-88e9ac9d-797c-41a0-a2c2-bf75f1424caf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f098b8fc390430b9f98879e6afcd1b7', 'request_id': 'req-88e9ac9d-797c-41a0-a2c2-bf75f1424caf'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:15.445390 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-88e9ac9d-797c-41a0-a2c2-bf75f1424caf admin admin] The token cache is being invalidate because project 6f098b8fc390430b9f98879e6afcd1b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:15.445688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-88e9ac9d-797c-41a0-a2c2-bf75f1424caf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:15.446637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-88e9ac9d-797c-41a0-a2c2-bf75f1424caf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:15.454323 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 870/1743] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:14 2022] DELETE /identity/v3/projects/6f098b8fc390430b9f98879e6afcd1b7 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:15.454323 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9ac9d-797c-41a0-a2c2-bf75f1424caf admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:15.461591 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-88e9ac9d-797c-41a0-a2c2-bf75f1424caf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:15.567471 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:15.583009 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df30cc7-86b9-416e-a14f-20c4a1774e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:15.586400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df30cc7-86b9-416e-a14f-20c4a1774e80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:15.587536 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df30cc7-86b9-416e-a14f-20c4a1774e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:15.588636 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df30cc7-86b9-416e-a14f-20c4a1774e80 admin admin] PATH_INFO: `/v3/users/3e24302094fc48e79c150c1b641a39af` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:15.594503 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df30cc7-86b9-416e-a14f-20c4a1774e80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e24302094fc48e79c150c1b641a39af)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:15.599538 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df30cc7-86b9-416e-a14f-20c4a1774e80 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:15.624491 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9ac9d-797c-41a0-a2c2-bf75f1424caf admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:15.639277 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29220f8e-57bc-41e3-9d63-d9d6c9fbdf33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:15.640176 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29220f8e-57bc-41e3-9d63-d9d6c9fbdf33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:15.640463 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29220f8e-57bc-41e3-9d63-d9d6c9fbdf33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:15.640740 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29220f8e-57bc-41e3-9d63-d9d6c9fbdf33 admin admin] PATH_INFO: `/v3/users/c135e6837b7b4d02963e5ae2e81c1aff` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:15.644078 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29220f8e-57bc-41e3-9d63-d9d6c9fbdf33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c135e6837b7b4d02963e5ae2e81c1aff)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:15.646115 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29220f8e-57bc-41e3-9d63-d9d6c9fbdf33 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:15.691851 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:15.857174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5df30cc7-86b9-416e-a14f-20c4a1774e80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e24302094fc48e79c150c1b641a39af'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:15.857550 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-29220f8e-57bc-41e3-9d63-d9d6c9fbdf33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c135e6837b7b4d02963e5ae2e81c1aff'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:15.857949 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5df30cc7-86b9-416e-a14f-20c4a1774e80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e24302094fc48e79c150c1b641a39af'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:15.858448 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-29220f8e-57bc-41e3-9d63-d9d6c9fbdf33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c135e6837b7b4d02963e5ae2e81c1aff'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:15.858644 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5df30cc7-86b9-416e-a14f-20c4a1774e80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e24302094fc48e79c150c1b641a39af'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:15.879585 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-29220f8e-57bc-41e3-9d63-d9d6c9fbdf33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c135e6837b7b4d02963e5ae2e81c1aff'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:15.880378 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-29220f8e-57bc-41e3-9d63-d9d6c9fbdf33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c135e6837b7b4d02963e5ae2e81c1aff'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:15.883508 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5df30cc7-86b9-416e-a14f-20c4a1774e80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e24302094fc48e79c150c1b641a39af'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:15.902151 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-29220f8e-57bc-41e3-9d63-d9d6c9fbdf33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c135e6837b7b4d02963e5ae2e81c1aff'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:15.907407 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5df30cc7-86b9-416e-a14f-20c4a1774e80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e24302094fc48e79c150c1b641a39af'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:15.942625 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 871/1744] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:38:15 2022] DELETE /identity/v3/users/c135e6837b7b4d02963e5ae2e81c1aff => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:15.943485 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29220f8e-57bc-41e3-9d63-d9d6c9fbdf33 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:15.956160 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-29220f8e-57bc-41e3-9d63-d9d6c9fbdf33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:15.974254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 874/1745] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:38:15 2022] DELETE /identity/v3/users/3e24302094fc48e79c150c1b641a39af => generated 0 bytes in 861 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:15.982387 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5deec11-c4a4-4915-b84c-170eb7f8f8c4 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:15.983086 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5deec11-c4a4-4915-b84c-170eb7f8f8c4 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:15.983555 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5deec11-c4a4-4915-b84c-170eb7f8f8c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:16.013649 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-b5deec11-c4a4-4915-b84c-170eb7f8f8c4 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:16.016454 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b5deec11-c4a4-4915-b84c-170eb7f8f8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:16.110124 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29220f8e-57bc-41e3-9d63-d9d6c9fbdf33 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:16.205039 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 875/1746] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:16.222597 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5deec11-c4a4-4915-b84c-170eb7f8f8c4 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:16.224728 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b5deec11-c4a4-4915-b84c-170eb7f8f8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:16.266380 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e069d02-2a44-4c96-988b-7a8e70346892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:16.267285 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e069d02-2a44-4c96-988b-7a8e70346892 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:16.267587 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e069d02-2a44-4c96-988b-7a8e70346892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:16.267831 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e069d02-2a44-4c96-988b-7a8e70346892 admin admin] PATH_INFO: `/v3/users/ac80c0e9b3404d588100672e2b25ed55` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:16.269123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e069d02-2a44-4c96-988b-7a8e70346892 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac80c0e9b3404d588100672e2b25ed55)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:16.272007 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e069d02-2a44-4c96-988b-7a8e70346892 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:16.279385 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:16.371010 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-2e069d02-2a44-4c96-988b-7a8e70346892 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac80c0e9b3404d588100672e2b25ed55'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:16.372673 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-2e069d02-2a44-4c96-988b-7a8e70346892 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac80c0e9b3404d588100672e2b25ed55'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:16.385050 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-2e069d02-2a44-4c96-988b-7a8e70346892 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac80c0e9b3404d588100672e2b25ed55'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:16.385785 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-2e069d02-2a44-4c96-988b-7a8e70346892 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac80c0e9b3404d588100672e2b25ed55'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:16.403170 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-2e069d02-2a44-4c96-988b-7a8e70346892 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac80c0e9b3404d588100672e2b25ed55'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:16.437515 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5deec11-c4a4-4915-b84c-170eb7f8f8c4 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:16.438949 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 872/1747] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:38:15 2022] DELETE /identity/v3/users/ac80c0e9b3404d588100672e2b25ed55 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:16.639183 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e6abf8-630b-4d08-945d-5454fbfb2c4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:16.639400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e6abf8-630b-4d08-945d-5454fbfb2c4b service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:16.639797 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e6abf8-630b-4d08-945d-5454fbfb2c4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:16.640844 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e6abf8-630b-4d08-945d-5454fbfb2c4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:16.644603 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a0e6abf8-630b-4d08-945d-5454fbfb2c4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:16.659777 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:16.769619 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e6abf8-630b-4d08-945d-5454fbfb2c4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:16.788178 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e6abf8-630b-4d08-945d-5454fbfb2c4b service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:16.812314 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 876/1748] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:38:17.154743 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e069d02-2a44-4c96-988b-7a8e70346892 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:17.156756 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-2e069d02-2a44-4c96-988b-7a8e70346892 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:17.180358 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e069d02-2a44-4c96-988b-7a8e70346892 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:17.189924 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d5eeaf-0fff-421c-ae66-7bd40acf50b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:17.191676 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d5eeaf-0fff-421c-ae66-7bd40acf50b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:17.193271 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d5eeaf-0fff-421c-ae66-7bd40acf50b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:17.194298 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d5eeaf-0fff-421c-ae66-7bd40acf50b9 admin admin] PATH_INFO: `/v3/projects/3b6fdee1681f41e388d25b044414b360` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:17.203387 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d5eeaf-0fff-421c-ae66-7bd40acf50b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b6fdee1681f41e388d25b044414b360)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:17.205841 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d5eeaf-0fff-421c-ae66-7bd40acf50b9 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:17.355785 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-e0d5eeaf-0fff-421c-ae66-7bd40acf50b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b6fdee1681f41e388d25b044414b360', 'request_id': 'req-e0d5eeaf-0fff-421c-ae66-7bd40acf50b9'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:17.377716 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-e0d5eeaf-0fff-421c-ae66-7bd40acf50b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b6fdee1681f41e388d25b044414b360', 'request_id': 'req-e0d5eeaf-0fff-421c-ae66-7bd40acf50b9'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:17.395350 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-e0d5eeaf-0fff-421c-ae66-7bd40acf50b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b6fdee1681f41e388d25b044414b360', 'request_id': 'req-e0d5eeaf-0fff-421c-ae66-7bd40acf50b9'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:17.431117 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-e0d5eeaf-0fff-421c-ae66-7bd40acf50b9 admin admin] The token cache is being invalidate because project 3b6fdee1681f41e388d25b044414b360 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:17.431394 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-e0d5eeaf-0fff-421c-ae66-7bd40acf50b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:17.434604 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-e0d5eeaf-0fff-421c-ae66-7bd40acf50b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:17.441522 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 873/1749] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:17 2022] DELETE /identity/v3/projects/3b6fdee1681f41e388d25b044414b360 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:17.446855 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9510f-06b4-4b02-a8e0-704c7204df9c None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:17.447352 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9510f-06b4-4b02-a8e0-704c7204df9c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:17.447454 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9510f-06b4-4b02-a8e0-704c7204df9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:17.486493 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-65d9510f-06b4-4b02-a8e0-704c7204df9c None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:17.487825 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-65d9510f-06b4-4b02-a8e0-704c7204df9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:17.597127 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 877/1750] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:17.605128 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef600c56-2fab-4a9d-9ab6-66fe840672c9 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:17.605447 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef600c56-2fab-4a9d-9ab6-66fe840672c9 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:17.605699 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef600c56-2fab-4a9d-9ab6-66fe840672c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:17.628884 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-ef600c56-2fab-4a9d-9ab6-66fe840672c9 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:17.630145 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-ef600c56-2fab-4a9d-9ab6-66fe840672c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:17.639180 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 874/1751] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:17.643680 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9510f-06b4-4b02-a8e0-704c7204df9c None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:17.655900 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9510f-06b4-4b02-a8e0-704c7204df9c None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:17.662215 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff971b-9033-4887-9ca6-003e9945313e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:17.663143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff971b-9033-4887-9ca6-003e9945313e admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:17.663658 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff971b-9033-4887-9ca6-003e9945313e admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:17.664609 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff971b-9033-4887-9ca6-003e9945313e admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:17.665597 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff971b-9033-4887-9ca6-003e9945313e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:38:17.666140 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff971b-9033-4887-9ca6-003e9945313e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:17.669867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff971b-9033-4887-9ca6-003e9945313e admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:17.716733 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 878/1752] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:38:17.718644 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef600c56-2fab-4a9d-9ab6-66fe840672c9 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:17.733893 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef600c56-2fab-4a9d-9ab6-66fe840672c9 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:17.768592 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa95e7-0523-4e92-8b60-4e0630fa04ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:17.769354 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa95e7-0523-4e92-8b60-4e0630fa04ed admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:17.769652 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa95e7-0523-4e92-8b60-4e0630fa04ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:17.769936 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa95e7-0523-4e92-8b60-4e0630fa04ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:17.770735 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aa95e7-0523-4e92-8b60-4e0630fa04ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:17.778734 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aa95e7-0523-4e92-8b60-4e0630fa04ed admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:17.847402 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 875/1753] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:17 2022] POST /identity/v3/projects => generated 387 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:17.848570 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff971b-9033-4887-9ca6-003e9945313e admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:17.862048 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff971b-9033-4887-9ca6-003e9945313e admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:18.008546 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f48ee-6562-4131-a384-8dbec2bd2a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:18.009403 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f48ee-6562-4131-a384-8dbec2bd2a8f admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:18.009537 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f48ee-6562-4131-a384-8dbec2bd2a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:18.009790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f48ee-6562-4131-a384-8dbec2bd2a8f admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:18.010838 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28f48ee-6562-4131-a384-8dbec2bd2a8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:18.013061 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28f48ee-6562-4131-a384-8dbec2bd2a8f admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:18.070810 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa95e7-0523-4e92-8b60-4e0630fa04ed admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:18.071862 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 879/1754] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:17 2022] POST /identity/v3/users => generated 294 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:18.082370 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa95e7-0523-4e92-8b60-4e0630fa04ed admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:18.088526 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e64010c-5926-477b-9644-294872cae44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:18.089017 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e64010c-5926-477b-9644-294872cae44d admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:18.089306 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e64010c-5926-477b-9644-294872cae44d admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:18.089593 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e64010c-5926-477b-9644-294872cae44d admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:18.090374 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e64010c-5926-477b-9644-294872cae44d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:18.092022 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e64010c-5926-477b-9644-294872cae44d admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:18.107818 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f48ee-6562-4131-a384-8dbec2bd2a8f admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:18.109011 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 876/1755] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:38:18.120591 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f48ee-6562-4131-a384-8dbec2bd2a8f admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:18.127682 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8017da-4927-4c6b-ad81-473c54522e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:18.128505 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8017da-4927-4c6b-ad81-473c54522e1c admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:18.128865 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8017da-4927-4c6b-ad81-473c54522e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:18.129291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8017da-4927-4c6b-ad81-473c54522e1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:18.130123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8017da-4927-4c6b-ad81-473c54522e1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:18.132425 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8017da-4927-4c6b-ad81-473c54522e1c admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:18.154206 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 880/1756] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:18.156465 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e64010c-5926-477b-9644-294872cae44d admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:18.178758 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e64010c-5926-477b-9644-294872cae44d admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:18.190462 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c7e06-df82-46d2-a8d2-ab81ffc6ea91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:18.192061 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c7e06-df82-46d2-a8d2-ab81ffc6ea91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:18.193172 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c7e06-df82-46d2-a8d2-ab81ffc6ea91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:18.193617 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c7e06-df82-46d2-a8d2-ab81ffc6ea91 admin admin] PATH_INFO: `/v3/projects/313e524e196d48c38fbfa9458a399728/users/3477d143046045248723add636d7cdcc/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:18.221264 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5850d50-edfc-42c9-bf91-d3b5f4423e28 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:18.221567 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5850d50-edfc-42c9-bf91-d3b5f4423e28 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:18.222185 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5850d50-edfc-42c9-bf91-d3b5f4423e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:18.225151 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278c7e06-df82-46d2-a8d2-ab81ffc6ea91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=313e524e196d48c38fbfa9458a399728, user_id=3477d143046045248723add636d7cdcc, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:18.227947 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278c7e06-df82-46d2-a8d2-ab81ffc6ea91 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:18.246309 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=79,keystone:DELETE=17,keystone:INSERT=4 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:18.249995 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-d5850d50-edfc-42c9-bf91-d3b5f4423e28 None None] MFA Rules not processed for user `e1635576dfe242f88ec1abb7be6359fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:18.252092 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d5850d50-edfc-42c9-bf91-d3b5f4423e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:18.252550 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 877/1757] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:18 2022] PUT /identity/v3/projects/313e524e196d48c38fbfa9458a399728/users/3477d143046045248723add636d7cdcc/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:38:18.263653 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca8e98-5c0f-4b56-a6b6-80a4fb627034 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:18.263938 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca8e98-5c0f-4b56-a6b6-80a4fb627034 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:18.264301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca8e98-5c0f-4b56-a6b6-80a4fb627034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:18.267486 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:18.317482 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-12ca8e98-5c0f-4b56-a6b6-80a4fb627034 None None] MFA Rules not processed for user `3477d143046045248723add636d7cdcc`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:18.319717 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-12ca8e98-5c0f-4b56-a6b6-80a4fb627034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:18.366368 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 881/1758] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:18.486441 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 878/1759] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:38:18.487748 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5850d50-edfc-42c9-bf91-d3b5f4423e28 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:18.616216 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5850d50-edfc-42c9-bf91-d3b5f4423e28 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:18.633333 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e992734d-1b05-4bb9-bf99-5b3bc68f7484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:18.638320 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e992734d-1b05-4bb9-bf99-5b3bc68f7484 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:18.638790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e992734d-1b05-4bb9-bf99-5b3bc68f7484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:18.639219 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e992734d-1b05-4bb9-bf99-5b3bc68f7484 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:18.640400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e992734d-1b05-4bb9-bf99-5b3bc68f7484 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:18.642801 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca8e98-5c0f-4b56-a6b6-80a4fb627034 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:18.643567 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e992734d-1b05-4bb9-bf99-5b3bc68f7484 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:18.646304 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-12ca8e98-5c0f-4b56-a6b6-80a4fb627034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:18.673137 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca8e98-5c0f-4b56-a6b6-80a4fb627034 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:18.692423 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 882/1760] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:18 2022] POST /identity/v3/projects => generated 387 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:38:18.704095 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8d78d-93b2-4788-bc05-59bf635902f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:18.709759 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8d78d-93b2-4788-bc05-59bf635902f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:18.712822 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e992734d-1b05-4bb9-bf99-5b3bc68f7484 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:18.713390 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8d78d-93b2-4788-bc05-59bf635902f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:18.713820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8d78d-93b2-4788-bc05-59bf635902f4 admin admin] PATH_INFO: `/v3/projects/ba45712ef62c42dbb408d059dac757c6` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:18.717594 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8d78d-93b2-4788-bc05-59bf635902f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba45712ef62c42dbb408d059dac757c6)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:18.719758 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8d78d-93b2-4788-bc05-59bf635902f4 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:18.843167 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-e4b8d78d-93b2-4788-bc05-59bf635902f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba45712ef62c42dbb408d059dac757c6', 'request_id': 'req-e4b8d78d-93b2-4788-bc05-59bf635902f4'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:18.855861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-e4b8d78d-93b2-4788-bc05-59bf635902f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba45712ef62c42dbb408d059dac757c6', 'request_id': 'req-e4b8d78d-93b2-4788-bc05-59bf635902f4'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:18.863500 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-e4b8d78d-93b2-4788-bc05-59bf635902f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba45712ef62c42dbb408d059dac757c6', 'request_id': 'req-e4b8d78d-93b2-4788-bc05-59bf635902f4'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:18.867040 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e992734d-1b05-4bb9-bf99-5b3bc68f7484 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:18.888153 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-e4b8d78d-93b2-4788-bc05-59bf635902f4 admin admin] The token cache is being invalidate because project ba45712ef62c42dbb408d059dac757c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:18.888726 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-e4b8d78d-93b2-4788-bc05-59bf635902f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:18.894578 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-e4b8d78d-93b2-4788-bc05-59bf635902f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:18.896843 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 879/1761] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:18 2022] DELETE /identity/v3/projects/ba45712ef62c42dbb408d059dac757c6 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:18.901089 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8d78d-93b2-4788-bc05-59bf635902f4 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:18.902724 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-e4b8d78d-93b2-4788-bc05-59bf635902f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:18.993309 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:19.022931 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8d78d-93b2-4788-bc05-59bf635902f4 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:19.028016 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a1ce21-f63d-4139-be3e-dbd6703f1f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:19.028920 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a1ce21-f63d-4139-be3e-dbd6703f1f2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:19.029210 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a1ce21-f63d-4139-be3e-dbd6703f1f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:19.029620 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a1ce21-f63d-4139-be3e-dbd6703f1f2e admin admin] PATH_INFO: `/v3/users/2878e3b2e93044ddb32cc40347ff9b2c` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:19.030975 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a1ce21-f63d-4139-be3e-dbd6703f1f2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2878e3b2e93044ddb32cc40347ff9b2c)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:19.033499 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a1ce21-f63d-4139-be3e-dbd6703f1f2e admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:19.040373 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b98c7c-91e8-41bd-afdd-ea0c502d5a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:19.041184 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b98c7c-91e8-41bd-afdd-ea0c502d5a0f admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:19.043140 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b98c7c-91e8-41bd-afdd-ea0c502d5a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:19.043534 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b98c7c-91e8-41bd-afdd-ea0c502d5a0f admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:19.044578 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b98c7c-91e8-41bd-afdd-ea0c502d5a0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:19.054185 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b98c7c-91e8-41bd-afdd-ea0c502d5a0f admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:19.063874 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:19.119924 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 883/1762] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:18 2022] POST /identity/v3/users => generated 300 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:19.125381 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b98c7c-91e8-41bd-afdd-ea0c502d5a0f admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:19.137045 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-73b98c7c-91e8-41bd-afdd-ea0c502d5a0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:19.155950 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b98c7c-91e8-41bd-afdd-ea0c502d5a0f admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:19.165613 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-15a1ce21-f63d-4139-be3e-dbd6703f1f2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2878e3b2e93044ddb32cc40347ff9b2c'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:19.166746 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-15a1ce21-f63d-4139-be3e-dbd6703f1f2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2878e3b2e93044ddb32cc40347ff9b2c'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:19.175473 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d67cba3-963e-420e-863e-8c532eeeb424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:19.176171 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67cba3-963e-420e-863e-8c532eeeb424 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:19.176453 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67cba3-963e-420e-863e-8c532eeeb424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:19.176743 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67cba3-963e-420e-863e-8c532eeeb424 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:19.177472 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d67cba3-963e-420e-863e-8c532eeeb424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:19.179127 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d67cba3-963e-420e-863e-8c532eeeb424 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:19.194712 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-15a1ce21-f63d-4139-be3e-dbd6703f1f2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2878e3b2e93044ddb32cc40347ff9b2c'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:19.195504 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-15a1ce21-f63d-4139-be3e-dbd6703f1f2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2878e3b2e93044ddb32cc40347ff9b2c'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:19.207256 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-15a1ce21-f63d-4139-be3e-dbd6703f1f2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2878e3b2e93044ddb32cc40347ff9b2c'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:19.209902 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 884/1763] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:19.210826 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d67cba3-963e-420e-863e-8c532eeeb424 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:19.216230 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6d67cba3-963e-420e-863e-8c532eeeb424 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:19.245800 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d67cba3-963e-420e-863e-8c532eeeb424 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:19.250188 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 880/1764] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:38:18 2022] DELETE /identity/v3/users/2878e3b2e93044ddb32cc40347ff9b2c => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:38:19.251948 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a1ce21-f63d-4139-be3e-dbd6703f1f2e admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:19.255812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-15a1ce21-f63d-4139-be3e-dbd6703f1f2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:19.364693 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a1ce21-f63d-4139-be3e-dbd6703f1f2e admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:19.374490 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c6605-4c04-4528-9460-0d426073b8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:19.375146 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c67abf-4534-4ca5-90c4-38da91d334ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:19.376299 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c67abf-4534-4ca5-90c4-38da91d334ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:19.376580 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c67abf-4534-4ca5-90c4-38da91d334ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:19.376698 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c6605-4c04-4528-9460-0d426073b8a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:19.376872 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c67abf-4534-4ca5-90c4-38da91d334ef admin admin] PATH_INFO: `/v3/projects/db667a1089aa45f8bbc26f89497ab07a` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:19.378076 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c6605-4c04-4528-9460-0d426073b8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:19.378393 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c6605-4c04-4528-9460-0d426073b8a2 admin admin] PATH_INFO: `/v3/projects/f2df8a0d89a248e6bb4ecb06a1849c05/users/24a399727a464da1ad91f05502637d7c/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:19.382815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c67abf-4534-4ca5-90c4-38da91d334ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=db667a1089aa45f8bbc26f89497ab07a)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:19.384938 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c67abf-4534-4ca5-90c4-38da91d334ef admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:19.418414 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c6605-4c04-4528-9460-0d426073b8a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2df8a0d89a248e6bb4ecb06a1849c05, user_id=24a399727a464da1ad91f05502637d7c, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:19.422006 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c6605-4c04-4528-9460-0d426073b8a2 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:19.464473 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 881/1765] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:19 2022] PUT /identity/v3/projects/f2df8a0d89a248e6bb4ecb06a1849c05/users/24a399727a464da1ad91f05502637d7c/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:19.468667 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c6605-4c04-4528-9460-0d426073b8a2 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:19.511937 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-b7c67abf-4534-4ca5-90c4-38da91d334ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db667a1089aa45f8bbc26f89497ab07a', 'request_id': 'req-b7c67abf-4534-4ca5-90c4-38da91d334ef'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:19.546793 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-b7c67abf-4534-4ca5-90c4-38da91d334ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db667a1089aa45f8bbc26f89497ab07a', 'request_id': 'req-b7c67abf-4534-4ca5-90c4-38da91d334ef'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:19.564233 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-b7c67abf-4534-4ca5-90c4-38da91d334ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db667a1089aa45f8bbc26f89497ab07a', 'request_id': 'req-b7c67abf-4534-4ca5-90c4-38da91d334ef'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:19.595010 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-b7c67abf-4534-4ca5-90c4-38da91d334ef admin admin] The token cache is being invalidate because project db667a1089aa45f8bbc26f89497ab07a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:19.596816 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-b7c67abf-4534-4ca5-90c4-38da91d334ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:19.597624 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-b7c67abf-4534-4ca5-90c4-38da91d334ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:19.603219 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 885/1766] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:19 2022] DELETE /identity/v3/projects/db667a1089aa45f8bbc26f89497ab07a => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:19.607955 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c8e86-949d-43e2-abc3-4619fa5d2c33 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:19.608258 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c8e86-949d-43e2-abc3-4619fa5d2c33 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:19.608713 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c8e86-949d-43e2-abc3-4619fa5d2c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:19.621234 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c6605-4c04-4528-9460-0d426073b8a2 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:19.639398 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c11c8e86-949d-43e2-abc3-4619fa5d2c33 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:19.645463 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c11c8e86-949d-43e2-abc3-4619fa5d2c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:19.655347 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:19.730913 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d227d-6ea6-4ae7-af50-04525495aea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:19.731689 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d227d-6ea6-4ae7-af50-04525495aea6 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:19.731961 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d227d-6ea6-4ae7-af50-04525495aea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:19.732236 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d227d-6ea6-4ae7-af50-04525495aea6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:19.733006 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233d227d-6ea6-4ae7-af50-04525495aea6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:19.734722 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233d227d-6ea6-4ae7-af50-04525495aea6 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:19.756014 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 882/1767] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:38:19.756961 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2221817-a1bd-4961-ac83-f492b866829c None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:19.757508 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2221817-a1bd-4961-ac83-f492b866829c None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:19.757929 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:19.758256 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2221817-a1bd-4961-ac83-f492b866829c None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:19.765862 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 883/1768] 10.222.0.15 () {54 vars in 921 bytes} [Wed Jun 15 19:38:19 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:38:19.767339 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2221817-a1bd-4961-ac83-f492b866829c None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:19.769477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-e2221817-a1bd-4961-ac83-f492b866829c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:19.791308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2221817-a1bd-4961-ac83-f492b866829c None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:19.797198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdd0c22-9ae3-49e7-befe-54f9f2285ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:19.797543 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 886/1769] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:38:19.798906 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd0c22-9ae3-49e7-befe-54f9f2285ebe admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:19.799408 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd0c22-9ae3-49e7-befe-54f9f2285ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:19.799849 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd0c22-9ae3-49e7-befe-54f9f2285ebe admin admin] PATH_INFO: `/v3/projects/f2df8a0d89a248e6bb4ecb06a1849c05/users/24a399727a464da1ad91f05502637d7c/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:19.801282 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f2837-bf5c-4edf-9911-be4938fa1850 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:19.801701 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f2837-bf5c-4edf-9911-be4938fa1850 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:19.802240 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f2837-bf5c-4edf-9911-be4938fa1850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:19.804641 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdd0c22-9ae3-49e7-befe-54f9f2285ebe admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2df8a0d89a248e6bb4ecb06a1849c05, user_id=24a399727a464da1ad91f05502637d7c, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:19.808900 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdd0c22-9ae3-49e7-befe-54f9f2285ebe admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:19.826576 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 884/1770] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:19 2022] PUT /identity/v3/projects/f2df8a0d89a248e6bb4ecb06a1849c05/users/24a399727a464da1ad91f05502637d7c/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:19.838358 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd90a4-79b3-4a62-b99c-b93241e0011f None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:19.838884 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd90a4-79b3-4a62-b99c-b93241e0011f None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:19.839306 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd90a4-79b3-4a62-b99c-b93241e0011f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:19.853700 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-420f2837-bf5c-4edf-9911-be4938fa1850 None None] MFA Rules not processed for user `3e74ac44e5814122a3226ac023f3688a`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:19.855205 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-420f2837-bf5c-4edf-9911-be4938fa1850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:19.862352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-1bfd90a4-79b3-4a62-b99c-b93241e0011f None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:19.865280 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-1bfd90a4-79b3-4a62-b99c-b93241e0011f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:19.962462 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 885/1771] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:19.964960 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb08ee-b651-4cd3-b164-1b32c26a7108 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:19.964960 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb08ee-b651-4cd3-b164-1b32c26a7108 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:19.965138 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb08ee-b651-4cd3-b164-1b32c26a7108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:20.004932 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 887/1772] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Jun 15 19:38:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:20.006060 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f2837-bf5c-4edf-9911-be4938fa1850 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:20.028702 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f2837-bf5c-4edf-9911-be4938fa1850 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:20.032712 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-2adb08ee-b651-4cd3-b164-1b32c26a7108 None None] MFA Rules not processed for user `24a399727a464da1ad91f05502637d7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:20.034668 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-2adb08ee-b651-4cd3-b164-1b32c26a7108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:20.040372 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a8037-7090-4339-b9e4-a2f936d288a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:20.041160 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a8037-7090-4339-b9e4-a2f936d288a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:20.042565 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a8037-7090-4339-b9e4-a2f936d288a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:20.043003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a8037-7090-4339-b9e4-a2f936d288a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:20.045753 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a8037-7090-4339-b9e4-a2f936d288a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:38:20.046075 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a8037-7090-4339-b9e4-a2f936d288a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:20.047951 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a8037-7090-4339-b9e4-a2f936d288a2 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:20.074592 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 888/1773] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:38:20.079766 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a8037-7090-4339-b9e4-a2f936d288a2 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:20.095735 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a8037-7090-4339-b9e4-a2f936d288a2 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:20.110031 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a53118b-4ec4-495b-81ca-b063ff51552b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:20.110917 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53118b-4ec4-495b-81ca-b063ff51552b admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:20.111306 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53118b-4ec4-495b-81ca-b063ff51552b admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:20.111727 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53118b-4ec4-495b-81ca-b063ff51552b admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:20.123857 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a53118b-4ec4-495b-81ca-b063ff51552b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:20.126206 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a53118b-4ec4-495b-81ca-b063ff51552b admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:20.170330 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 886/1774] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:19 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:20.186032 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adb08ee-b651-4cd3-b164-1b32c26a7108 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:20.188037 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-2adb08ee-b651-4cd3-b164-1b32c26a7108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:20.202013 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 889/1775] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:20 2022] POST /identity/v3/projects => generated 365 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:20.203881 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a53118b-4ec4-495b-81ca-b063ff51552b admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:20.206722 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7a53118b-4ec4-495b-81ca-b063ff51552b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:20.231892 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adb08ee-b651-4cd3-b164-1b32c26a7108 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:20.294556 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:20.316307 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a53118b-4ec4-495b-81ca-b063ff51552b admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:20.324459 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac4494-5dfc-453f-ae57-927cb1ef6d31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:20.325187 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac4494-5dfc-453f-ae57-927cb1ef6d31 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:20.325521 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac4494-5dfc-453f-ae57-927cb1ef6d31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:20.325803 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac4494-5dfc-453f-ae57-927cb1ef6d31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:20.388485 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d17a36-e7ca-488b-b7c2-9b721f6b43fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:20.389054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d17a36-e7ca-488b-b7c2-9b721f6b43fb service nova] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:20.391356 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d17a36-e7ca-488b-b7c2-9b721f6b43fb service nova] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:20.391641 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d17a36-e7ca-488b-b7c2-9b721f6b43fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:20.415701 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d17a36-e7ca-488b-b7c2-9b721f6b43fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:20.418657 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d17a36-e7ca-488b-b7c2-9b721f6b43fb service nova] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:20.459866 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 887/1776] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:38:20.461464 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d17a36-e7ca-488b-b7c2-9b721f6b43fb service nova] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:20.477631 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfac4494-5dfc-453f-ae57-927cb1ef6d31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:20.489883 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfac4494-5dfc-453f-ae57-927cb1ef6d31 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:20.515816 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 890/1777] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:20 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:38:20.642071 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d17a36-e7ca-488b-b7c2-9b721f6b43fb service nova] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:20.647854 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2854266-51fb-4840-9530-d60ac1e235c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:20.648793 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2854266-51fb-4840-9530-d60ac1e235c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:20.649283 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2854266-51fb-4840-9530-d60ac1e235c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:20.649699 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2854266-51fb-4840-9530-d60ac1e235c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:20.650586 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2854266-51fb-4840-9530-d60ac1e235c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:20.653333 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2854266-51fb-4840-9530-d60ac1e235c1 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:20.704873 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 888/1778] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:20 2022] POST /identity/v3/users => generated 283 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:20.706053 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac4494-5dfc-453f-ae57-927cb1ef6d31 service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:20.723714 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2854266-51fb-4840-9530-d60ac1e235c1 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:20.724916 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:20.741285 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac4494-5dfc-453f-ae57-927cb1ef6d31 service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:20.754559 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27b646-b0b4-4879-8fa9-367050d52f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:20.758280 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27b646-b0b4-4879-8fa9-367050d52f44 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:20.758280 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27b646-b0b4-4879-8fa9-367050d52f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:20.761237 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27b646-b0b4-4879-8fa9-367050d52f44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:20.762122 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef27b646-b0b4-4879-8fa9-367050d52f44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:20.762648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2854266-51fb-4840-9530-d60ac1e235c1 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:20.771783 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef27b646-b0b4-4879-8fa9-367050d52f44 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:20.779559 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8080033-3f13-469e-bc6e-e5693c5e8d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:20.781449 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8080033-3f13-469e-bc6e-e5693c5e8d0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:20.781721 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8080033-3f13-469e-bc6e-e5693c5e8d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:20.783106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8080033-3f13-469e-bc6e-e5693c5e8d0c admin admin] PATH_INFO: `/v3/users/3477d143046045248723add636d7cdcc` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:20.790329 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8080033-3f13-469e-bc6e-e5693c5e8d0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3477d143046045248723add636d7cdcc)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:20.798879 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8080033-3f13-469e-bc6e-e5693c5e8d0c admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:20.818995 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 891/1779] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:20.820152 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27b646-b0b4-4879-8fa9-367050d52f44 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:20.852473 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27b646-b0b4-4879-8fa9-367050d52f44 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:20.873402 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff9375-bf2c-42cc-957c-0bb2e2d94dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:20.874796 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff9375-bf2c-42cc-957c-0bb2e2d94dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:20.875091 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff9375-bf2c-42cc-957c-0bb2e2d94dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:20.875370 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff9375-bf2c-42cc-957c-0bb2e2d94dc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:20.885727 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff9375-bf2c-42cc-957c-0bb2e2d94dc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:20.890626 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff9375-bf2c-42cc-957c-0bb2e2d94dc1 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:20.922353 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 892/1780] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:38:20.926872 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff9375-bf2c-42cc-957c-0bb2e2d94dc1 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:20.940545 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff9375-bf2c-42cc-957c-0bb2e2d94dc1 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:20.948763 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff43aa3-23bf-4ba5-8a0e-c1c195b93035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:20.950139 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff43aa3-23bf-4ba5-8a0e-c1c195b93035 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:20.950430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff43aa3-23bf-4ba5-8a0e-c1c195b93035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:20.952048 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff43aa3-23bf-4ba5-8a0e-c1c195b93035 admin admin] PATH_INFO: `/v3/projects/3824390d416742efab487d3d676f468c/users/87653066617d49bab8dc39e8cafd0e7b/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:20.956622 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-d8080033-3f13-469e-bc6e-e5693c5e8d0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3477d143046045248723add636d7cdcc'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:20.957697 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-d8080033-3f13-469e-bc6e-e5693c5e8d0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3477d143046045248723add636d7cdcc'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:20.972455 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-d8080033-3f13-469e-bc6e-e5693c5e8d0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3477d143046045248723add636d7cdcc'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:20.974850 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-d8080033-3f13-469e-bc6e-e5693c5e8d0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3477d143046045248723add636d7cdcc'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:20.998347 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff43aa3-23bf-4ba5-8a0e-c1c195b93035 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3824390d416742efab487d3d676f468c, user_id=87653066617d49bab8dc39e8cafd0e7b, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:21.004756 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff43aa3-23bf-4ba5-8a0e-c1c195b93035 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:21.014868 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-d8080033-3f13-469e-bc6e-e5693c5e8d0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3477d143046045248723add636d7cdcc'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:21.038148 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 893/1781] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:20 2022] PUT /identity/v3/projects/3824390d416742efab487d3d676f468c/users/87653066617d49bab8dc39e8cafd0e7b/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:38:21.045626 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 889/1782] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:38:20 2022] DELETE /identity/v3/users/3477d143046045248723add636d7cdcc => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:21.048545 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98e214-cfc9-42e7-900a-3f0023286e7c None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:21.048844 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98e214-cfc9-42e7-900a-3f0023286e7c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:21.049138 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98e214-cfc9-42e7-900a-3f0023286e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:21.052683 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01937776-e8cb-44bc-b22a-96cb9690f204 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:21.052980 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01937776-e8cb-44bc-b22a-96cb9690f204 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:21.053258 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01937776-e8cb-44bc-b22a-96cb9690f204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:21.073118 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-3a98e214-cfc9-42e7-900a-3f0023286e7c None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:21.074471 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3a98e214-cfc9-42e7-900a-3f0023286e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:21.108458 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-01937776-e8cb-44bc-b22a-96cb9690f204 None None] MFA Rules not processed for user `87653066617d49bab8dc39e8cafd0e7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:21.109793 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-01937776-e8cb-44bc-b22a-96cb9690f204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:21.116273 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:21.188805 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 894/1783] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:21.210151 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98e214-cfc9-42e7-900a-3f0023286e7c None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:21.212389 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3a98e214-cfc9-42e7-900a-3f0023286e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:21.251086 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 890/1784] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:21 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:21.265290 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01937776-e8cb-44bc-b22a-96cb9690f204 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:21.282259 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-01937776-e8cb-44bc-b22a-96cb9690f204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:21.293751 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98e214-cfc9-42e7-900a-3f0023286e7c None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:21.301396 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11804f19-bf31-470a-bb75-43c83f261190 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:21.303754 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11804f19-bf31-470a-bb75-43c83f261190 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:21.303899 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11804f19-bf31-470a-bb75-43c83f261190 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:21.304191 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11804f19-bf31-470a-bb75-43c83f261190 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:21.315318 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01937776-e8cb-44bc-b22a-96cb9690f204 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:21.320879 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6901de2-430e-4ac1-adb2-e175cd0a16f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:21.321554 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6901de2-430e-4ac1-adb2-e175cd0a16f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:21.321834 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6901de2-430e-4ac1-adb2-e175cd0a16f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:21.322252 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6901de2-430e-4ac1-adb2-e175cd0a16f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:21.322882 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6901de2-430e-4ac1-adb2-e175cd0a16f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:21.325382 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6901de2-430e-4ac1-adb2-e175cd0a16f7 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:21.329961 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11804f19-bf31-470a-bb75-43c83f261190 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:21.331694 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11804f19-bf31-470a-bb75-43c83f261190 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:21.370115 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 895/1785] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:38:21.383249 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 891/1786] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:21 2022] POST /identity/v3/projects => generated 365 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:21.387171 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11804f19-bf31-470a-bb75-43c83f261190 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:21.410747 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11804f19-bf31-470a-bb75-43c83f261190 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:21.517729 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b5744-dfb6-402f-83bc-c4489466139c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:21.518597 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b5744-dfb6-402f-83bc-c4489466139c admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:21.519220 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b5744-dfb6-402f-83bc-c4489466139c admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:21.519220 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b5744-dfb6-402f-83bc-c4489466139c admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:21.519999 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b5744-dfb6-402f-83bc-c4489466139c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:21.521860 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b5744-dfb6-402f-83bc-c4489466139c admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:21.529176 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:21.561366 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 896/1787] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:21 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:21.562721 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6901de2-430e-4ac1-adb2-e175cd0a16f7 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:21.577643 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6901de2-430e-4ac1-adb2-e175cd0a16f7 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:21.583593 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f686612f-f30e-4a29-8e38-e72a5be48409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:21.585013 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686612f-f30e-4a29-8e38-e72a5be48409 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:21.585534 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686612f-f30e-4a29-8e38-e72a5be48409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:21.585914 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686612f-f30e-4a29-8e38-e72a5be48409 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:21.586824 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f686612f-f30e-4a29-8e38-e72a5be48409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:21.588786 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f686612f-f30e-4a29-8e38-e72a5be48409 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:21.607378 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 892/1788] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:21 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:21.608937 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b5744-dfb6-402f-83bc-c4489466139c admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:21.620957 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b5744-dfb6-402f-83bc-c4489466139c admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:21.628961 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d530ac7b-33f2-4e68-9258-10d454f42ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:21.630165 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530ac7b-33f2-4e68-9258-10d454f42ae8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:21.630462 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530ac7b-33f2-4e68-9258-10d454f42ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:21.630749 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530ac7b-33f2-4e68-9258-10d454f42ae8 admin admin] PATH_INFO: `/v3/projects/736cecd0d293468db2803ea4c1aa15e8/users/069b950d675343d3a0e8dc393f49d3c8/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:21.654042 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d530ac7b-33f2-4e68-9258-10d454f42ae8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=736cecd0d293468db2803ea4c1aa15e8, user_id=069b950d675343d3a0e8dc393f49d3c8, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:21.690773 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d530ac7b-33f2-4e68-9258-10d454f42ae8 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:21.715922 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 897/1789] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:21 2022] PUT /identity/v3/projects/736cecd0d293468db2803ea4c1aa15e8/users/069b950d675343d3a0e8dc393f49d3c8/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:38:21.720975 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f686612f-f30e-4a29-8e38-e72a5be48409 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:21.749444 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f686612f-f30e-4a29-8e38-e72a5be48409 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:21.843005 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83343935-5e7b-41da-9282-e9e741403b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:21.843755 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83343935-5e7b-41da-9282-e9e741403b74 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:21.844046 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83343935-5e7b-41da-9282-e9e741403b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:21.844328 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83343935-5e7b-41da-9282-e9e741403b74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:21.845136 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83343935-5e7b-41da-9282-e9e741403b74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:21.847864 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83343935-5e7b-41da-9282-e9e741403b74 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:21.867115 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d530ac7b-33f2-4e68-9258-10d454f42ae8 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:21.867683 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 893/1790] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:21 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:21.872992 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d530ac7b-33f2-4e68-9258-10d454f42ae8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:21.878651 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83343935-5e7b-41da-9282-e9e741403b74 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:21.897036 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d530ac7b-33f2-4e68-9258-10d454f42ae8 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:21.906864 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83343935-5e7b-41da-9282-e9e741403b74 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:21.911978 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f101d20-5d47-48e1-9aa5-020861bb7a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:21.913116 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f101d20-5d47-48e1-9aa5-020861bb7a29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:21.914136 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f101d20-5d47-48e1-9aa5-020861bb7a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:21.915012 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f101d20-5d47-48e1-9aa5-020861bb7a29 admin admin] PATH_INFO: `/v3/projects/77922c50e37b48d4907d81074515a392` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:21.918392 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f101d20-5d47-48e1-9aa5-020861bb7a29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77922c50e37b48d4907d81074515a392)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:21.919285 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1767bc85-a68a-4a2e-9694-e18a76363883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:21.923284 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f101d20-5d47-48e1-9aa5-020861bb7a29 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:21.924025 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767bc85-a68a-4a2e-9694-e18a76363883 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:21.924462 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767bc85-a68a-4a2e-9694-e18a76363883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:21.924840 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767bc85-a68a-4a2e-9694-e18a76363883 admin admin] PATH_INFO: `/v3/projects/736cecd0d293468db2803ea4c1aa15e8/users/069b950d675343d3a0e8dc393f49d3c8/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:21.929205 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1767bc85-a68a-4a2e-9694-e18a76363883 admin admin] RBAC: Authorizing `identity:create_grant(project_id=736cecd0d293468db2803ea4c1aa15e8, user_id=069b950d675343d3a0e8dc393f49d3c8, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:21.940754 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1767bc85-a68a-4a2e-9694-e18a76363883 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:21.958670 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 894/1791] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:21 2022] PUT /identity/v3/projects/736cecd0d293468db2803ea4c1aa15e8/users/069b950d675343d3a0e8dc393f49d3c8/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:21.970901 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ef32c-b015-400b-9c7a-bf763793ce95 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:21.971432 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ef32c-b015-400b-9c7a-bf763793ce95 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:21.971811 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ef32c-b015-400b-9c7a-bf763793ce95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:22.029416 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-752ef32c-b015-400b-9c7a-bf763793ce95 None None] MFA Rules not processed for user `069b950d675343d3a0e8dc393f49d3c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:22.031711 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-752ef32c-b015-400b-9c7a-bf763793ce95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:22.035999 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8f101d20-5d47-48e1-9aa5-020861bb7a29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77922c50e37b48d4907d81074515a392', 'request_id': 'req-8f101d20-5d47-48e1-9aa5-020861bb7a29'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:22.052003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8f101d20-5d47-48e1-9aa5-020861bb7a29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77922c50e37b48d4907d81074515a392', 'request_id': 'req-8f101d20-5d47-48e1-9aa5-020861bb7a29'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:22.067900 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8f101d20-5d47-48e1-9aa5-020861bb7a29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77922c50e37b48d4907d81074515a392', 'request_id': 'req-8f101d20-5d47-48e1-9aa5-020861bb7a29'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:22.090517 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:22.097820 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8f101d20-5d47-48e1-9aa5-020861bb7a29 admin admin] The token cache is being invalidate because project 77922c50e37b48d4907d81074515a392 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:22.098531 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8f101d20-5d47-48e1-9aa5-020861bb7a29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:22.099409 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8f101d20-5d47-48e1-9aa5-020861bb7a29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:22.102249 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 898/1792] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:21 2022] DELETE /identity/v3/projects/77922c50e37b48d4907d81074515a392 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:22.110031 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a72a15-f6ff-4a3c-af99-155bd81ec03e None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:22.110495 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a72a15-f6ff-4a3c-af99-155bd81ec03e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:22.110917 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a72a15-f6ff-4a3c-af99-155bd81ec03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:22.123995 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 895/1793] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:21 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:22.139409 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-23a72a15-f6ff-4a3c-af99-155bd81ec03e None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:22.140322 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752ef32c-b015-400b-9c7a-bf763793ce95 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:22.141108 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-23a72a15-f6ff-4a3c-af99-155bd81ec03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:22.141994 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-752ef32c-b015-400b-9c7a-bf763793ce95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:22.244407 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752ef32c-b015-400b-9c7a-bf763793ce95 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:22.252938 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac9b641-2255-455e-a21b-8ced363b2f9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:22.254072 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9b641-2255-455e-a21b-8ced363b2f9f service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:22.254630 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9b641-2255-455e-a21b-8ced363b2f9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:22.255250 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9b641-2255-455e-a21b-8ced363b2f9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:22.264805 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 899/1794] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:22.269976 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308fafc-5012-444b-9c1d-81f3a2d5ee7b None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:22.270579 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac9b641-2255-455e-a21b-8ced363b2f9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:22.272098 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac9b641-2255-455e-a21b-8ced363b2f9f service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:22.272824 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308fafc-5012-444b-9c1d-81f3a2d5ee7b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:22.273381 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308fafc-5012-444b-9c1d-81f3a2d5ee7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:22.288884 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 896/1795] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:22 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:38:22.325264 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-0308fafc-5012-444b-9c1d-81f3a2d5ee7b None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:22.339444 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0308fafc-5012-444b-9c1d-81f3a2d5ee7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:22.384860 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 900/1796] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:22.390911 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac9b641-2255-455e-a21b-8ced363b2f9f service cinder] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:22.424458 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac9b641-2255-455e-a21b-8ced363b2f9f service cinder] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:22.436078 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da1c33-ad20-4316-b279-e8d229492e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:22.436912 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da1c33-ad20-4316-b279-e8d229492e5d admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:22.438658 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da1c33-ad20-4316-b279-e8d229492e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:22.439044 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da1c33-ad20-4316-b279-e8d229492e5d admin admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:22.441437 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30da1c33-ad20-4316-b279-e8d229492e5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:38:22.443758 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30da1c33-ad20-4316-b279-e8d229492e5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:22.445481 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30da1c33-ad20-4316-b279-e8d229492e5d admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:22.453941 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0308fafc-5012-444b-9c1d-81f3a2d5ee7b None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:22.479130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0308fafc-5012-444b-9c1d-81f3a2d5ee7b None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:22.485315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 897/1797] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:38:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:38:22.489358 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da1c33-ad20-4316-b279-e8d229492e5d admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:22.495302 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f145f288-e372-4d82-8642-f085570480ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:22.496121 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145f288-e372-4d82-8642-f085570480ae service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:22.496241 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145f288-e372-4d82-8642-f085570480ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:22.496583 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145f288-e372-4d82-8642-f085570480ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:22.503080 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f145f288-e372-4d82-8642-f085570480ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:22.524476 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da1c33-ad20-4316-b279-e8d229492e5d admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:22.533578 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382cfca5-6d2b-4ee4-bec9-3adec3645d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:22.534430 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382cfca5-6d2b-4ee4-bec9-3adec3645d3d admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:22.534814 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382cfca5-6d2b-4ee4-bec9-3adec3645d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:22.535180 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382cfca5-6d2b-4ee4-bec9-3adec3645d3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:22.536035 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382cfca5-6d2b-4ee4-bec9-3adec3645d3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:22.538078 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382cfca5-6d2b-4ee4-bec9-3adec3645d3d admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:22.580451 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 898/1798] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:22 2022] POST /identity/v3/projects => generated 381 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:22.584767 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382cfca5-6d2b-4ee4-bec9-3adec3645d3d admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:22.625497 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:22.707337 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f145f288-e372-4d82-8642-f085570480ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:22.708920 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f145f288-e372-4d82-8642-f085570480ae service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:22.756443 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 901/1799] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:22 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:38:22.797005 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382cfca5-6d2b-4ee4-bec9-3adec3645d3d admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:22.807183 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4851c-7fb0-4c6c-b75b-5ceb10884b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:22.808279 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4851c-7fb0-4c6c-b75b-5ceb10884b90 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:22.808570 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4851c-7fb0-4c6c-b75b-5ceb10884b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:22.808852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4851c-7fb0-4c6c-b75b-5ceb10884b90 admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:22.821343 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e4851c-7fb0-4c6c-b75b-5ceb10884b90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:22.826593 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e4851c-7fb0-4c6c-b75b-5ceb10884b90 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:22.889028 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 899/1800] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:22 2022] POST /identity/v3/users => generated 291 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:38:22.893254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f145f288-e372-4d82-8642-f085570480ae service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:22.905425 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f145f288-e372-4d82-8642-f085570480ae service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:22.910509 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb83a4-ede1-4406-8ca7-6e1692a1f04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:22.911250 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb83a4-ede1-4406-8ca7-6e1692a1f04e admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:22.911764 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb83a4-ede1-4406-8ca7-6e1692a1f04e admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:22.912255 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb83a4-ede1-4406-8ca7-6e1692a1f04e admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:22.912982 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb83a4-ede1-4406-8ca7-6e1692a1f04e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:22.914689 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb83a4-ede1-4406-8ca7-6e1692a1f04e admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:22.932139 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 902/1801] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:22 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:38:22.934503 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4851c-7fb0-4c6c-b75b-5ceb10884b90 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:22.971252 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4851c-7fb0-4c6c-b75b-5ceb10884b90 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:22.980233 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de6135c-5a26-4dd7-954f-b802dda94ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:22.981179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6135c-5a26-4dd7-954f-b802dda94ac3 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:22.981652 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6135c-5a26-4dd7-954f-b802dda94ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:22.981951 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6135c-5a26-4dd7-954f-b802dda94ac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:22.983150 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de6135c-5a26-4dd7-954f-b802dda94ac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:22.984793 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de6135c-5a26-4dd7-954f-b802dda94ac3 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:23.004252 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 900/1802] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:22 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:23.005658 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb83a4-ede1-4406-8ca7-6e1692a1f04e admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:23.020319 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb83a4-ede1-4406-8ca7-6e1692a1f04e admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:23.028555 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2f159-4b4f-4378-852b-2bee6a7fd9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:23.031155 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2f159-4b4f-4378-852b-2bee6a7fd9d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:23.031547 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2f159-4b4f-4378-852b-2bee6a7fd9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:23.033396 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2f159-4b4f-4378-852b-2bee6a7fd9d3 admin admin] PATH_INFO: `/v3/projects/55c213aa313b4afb8bdaedcde44a732b/users/fc43ce0c9a8f4595919d447f1f9411c2/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:23.060263 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f2f159-4b4f-4378-852b-2bee6a7fd9d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55c213aa313b4afb8bdaedcde44a732b, user_id=fc43ce0c9a8f4595919d447f1f9411c2, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:23.064154 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f2f159-4b4f-4378-852b-2bee6a7fd9d3 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:23.084337 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 903/1803] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:23 2022] PUT /identity/v3/projects/55c213aa313b4afb8bdaedcde44a732b/users/fc43ce0c9a8f4595919d447f1f9411c2/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:23.096261 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074ec6e-5303-41eb-9fa7-ad9d1d859976 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:23.096551 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074ec6e-5303-41eb-9fa7-ad9d1d859976 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:23.097098 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074ec6e-5303-41eb-9fa7-ad9d1d859976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:23.131961 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-6074ec6e-5303-41eb-9fa7-ad9d1d859976 None None] MFA Rules not processed for user `fc43ce0c9a8f4595919d447f1f9411c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:23.133286 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-6074ec6e-5303-41eb-9fa7-ad9d1d859976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:23.186127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:23.233388 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2f159-4b4f-4378-852b-2bee6a7fd9d3 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:23.240961 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-80f2f159-4b4f-4378-852b-2bee6a7fd9d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:23.277022 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 901/1804] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:23.278080 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074ec6e-5303-41eb-9fa7-ad9d1d859976 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:23.395466 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074ec6e-5303-41eb-9fa7-ad9d1d859976 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:23.400642 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2f159-4b4f-4378-852b-2bee6a7fd9d3 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:23.410944 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc03878-7aa8-4035-9fea-1dcc0f524797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:23.411786 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc03878-7aa8-4035-9fea-1dcc0f524797 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:23.412069 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc03878-7aa8-4035-9fea-1dcc0f524797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:23.412345 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc03878-7aa8-4035-9fea-1dcc0f524797 admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:23.413097 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc03878-7aa8-4035-9fea-1dcc0f524797 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:23.414076 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03408026-819e-4292-9beb-014ccd3328ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:23.415112 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03408026-819e-4292-9beb-014ccd3328ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:23.415492 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03408026-819e-4292-9beb-014ccd3328ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:23.417766 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03408026-819e-4292-9beb-014ccd3328ac admin admin] PATH_INFO: `/v3/projects/313e524e196d48c38fbfa9458a399728` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:23.419725 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc03878-7aa8-4035-9fea-1dcc0f524797 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:23.422435 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03408026-819e-4292-9beb-014ccd3328ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=313e524e196d48c38fbfa9458a399728)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:23.424503 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03408026-819e-4292-9beb-014ccd3328ac admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:23.476340 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 902/1805] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:23 2022] POST /identity/v3/projects => generated 383 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:23.482399 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc03878-7aa8-4035-9fea-1dcc0f524797 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:23.503883 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-03408026-819e-4292-9beb-014ccd3328ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '313e524e196d48c38fbfa9458a399728', 'request_id': 'req-03408026-819e-4292-9beb-014ccd3328ac'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:23.518067 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-03408026-819e-4292-9beb-014ccd3328ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '313e524e196d48c38fbfa9458a399728', 'request_id': 'req-03408026-819e-4292-9beb-014ccd3328ac'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:23.528666 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-03408026-819e-4292-9beb-014ccd3328ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '313e524e196d48c38fbfa9458a399728', 'request_id': 'req-03408026-819e-4292-9beb-014ccd3328ac'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:23.553697 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-03408026-819e-4292-9beb-014ccd3328ac admin admin] The token cache is being invalidate because project 313e524e196d48c38fbfa9458a399728 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:23.554060 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-03408026-819e-4292-9beb-014ccd3328ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:23.554936 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-03408026-819e-4292-9beb-014ccd3328ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:23.557077 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 904/1806] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:23 2022] DELETE /identity/v3/projects/313e524e196d48c38fbfa9458a399728 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:23.563183 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03408026-819e-4292-9beb-014ccd3328ac admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:23.567350 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-03408026-819e-4292-9beb-014ccd3328ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:23.625108 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:23.641263 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc03878-7aa8-4035-9fea-1dcc0f524797 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:23.655928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13147782-e6e9-4191-b6a2-6d6047f97578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:23.656630 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13147782-e6e9-4191-b6a2-6d6047f97578 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:23.656917 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13147782-e6e9-4191-b6a2-6d6047f97578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:23.657194 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13147782-e6e9-4191-b6a2-6d6047f97578 admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:23.657952 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13147782-e6e9-4191-b6a2-6d6047f97578 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:23.659941 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13147782-e6e9-4191-b6a2-6d6047f97578 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:23.660819 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03408026-819e-4292-9beb-014ccd3328ac admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:23.665835 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a8e6e-d648-4793-9fba-66a57cd1315a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:23.666754 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a8e6e-d648-4793-9fba-66a57cd1315a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:23.667036 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a8e6e-d648-4793-9fba-66a57cd1315a admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:23.667312 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a8e6e-d648-4793-9fba-66a57cd1315a admin admin] PATH_INFO: `/v3/users/24a399727a464da1ad91f05502637d7c` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:23.669059 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47a8e6e-d648-4793-9fba-66a57cd1315a admin admin] RBAC: Authorizing `identity:delete_user(user_id=24a399727a464da1ad91f05502637d7c)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:23.671043 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47a8e6e-d648-4793-9fba-66a57cd1315a admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:23.707056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 903/1807] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:23 2022] POST /identity/v3/users => generated 298 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:38:23.710540 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13147782-e6e9-4191-b6a2-6d6047f97578 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:23.712442 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-13147782-e6e9-4191-b6a2-6d6047f97578 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:23.729059 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13147782-e6e9-4191-b6a2-6d6047f97578 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:23.735248 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab9649-a7ef-45a4-bf01-831500792d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:23.735928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab9649-a7ef-45a4-bf01-831500792d67 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:23.736560 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab9649-a7ef-45a4-bf01-831500792d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:23.736843 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab9649-a7ef-45a4-bf01-831500792d67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:23.737613 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab9649-a7ef-45a4-bf01-831500792d67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:23.739714 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-a47a8e6e-d648-4793-9fba-66a57cd1315a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24a399727a464da1ad91f05502637d7c'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:23.740712 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-a47a8e6e-d648-4793-9fba-66a57cd1315a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24a399727a464da1ad91f05502637d7c'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:23.741286 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-a47a8e6e-d648-4793-9fba-66a57cd1315a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24a399727a464da1ad91f05502637d7c'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:23.746330 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab9649-a7ef-45a4-bf01-831500792d67 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:23.757091 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-a47a8e6e-d648-4793-9fba-66a57cd1315a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24a399727a464da1ad91f05502637d7c'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:23.765680 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 904/1808] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:23 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:38:23.766766 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab9649-a7ef-45a4-bf01-831500792d67 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:23.767061 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-a47a8e6e-d648-4793-9fba-66a57cd1315a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24a399727a464da1ad91f05502637d7c'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:23.769095 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-8aab9649-a7ef-45a4-bf01-831500792d67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:23.786053 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab9649-a7ef-45a4-bf01-831500792d67 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:23.786599 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 905/1809] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:38:23 2022] DELETE /identity/v3/users/24a399727a464da1ad91f05502637d7c => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:38:23.833641 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:23.903738 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85a839d-698a-4678-80a1-e1f152f2129f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:23.904708 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a839d-698a-4678-80a1-e1f152f2129f admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:23.904997 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a839d-698a-4678-80a1-e1f152f2129f admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:23.905283 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a839d-698a-4678-80a1-e1f152f2129f admin admin] PATH_INFO: `/v3/projects/fb504803729b497ea40a80384946d75e/users/ef5d0b8c3bc145919e19b8a44da4e6d9/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:23.922583 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85a839d-698a-4678-80a1-e1f152f2129f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb504803729b497ea40a80384946d75e, user_id=ef5d0b8c3bc145919e19b8a44da4e6d9, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:23.925289 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85a839d-698a-4678-80a1-e1f152f2129f admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:23.942762 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 905/1810] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:23 2022] PUT /identity/v3/projects/fb504803729b497ea40a80384946d75e/users/ef5d0b8c3bc145919e19b8a44da4e6d9/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:23.946718 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a8e6e-d648-4793-9fba-66a57cd1315a admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:23.957117 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a8e6e-d648-4793-9fba-66a57cd1315a admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:24.021261 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b90137-6390-4680-a81f-bbb93d9f96d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:24.021947 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b90137-6390-4680-a81f-bbb93d9f96d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:24.022443 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b90137-6390-4680-a81f-bbb93d9f96d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:24.022760 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b90137-6390-4680-a81f-bbb93d9f96d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:24.023758 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b90137-6390-4680-a81f-bbb93d9f96d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:24.025285 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b90137-6390-4680-a81f-bbb93d9f96d4 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:24.037522 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 906/1811] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:23 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:24.041775 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85a839d-698a-4678-80a1-e1f152f2129f admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:24.057027 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85a839d-698a-4678-80a1-e1f152f2129f admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:24.070109 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db387a58-67c9-48d4-bedb-ed1bb11f70e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:24.071050 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db387a58-67c9-48d4-bedb-ed1bb11f70e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:24.071332 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db387a58-67c9-48d4-bedb-ed1bb11f70e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:24.071607 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db387a58-67c9-48d4-bedb-ed1bb11f70e3 admin admin] PATH_INFO: `/v3/projects/fb504803729b497ea40a80384946d75e/users/ef5d0b8c3bc145919e19b8a44da4e6d9/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:24.073860 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db387a58-67c9-48d4-bedb-ed1bb11f70e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb504803729b497ea40a80384946d75e, user_id=ef5d0b8c3bc145919e19b8a44da4e6d9, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:24.076656 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db387a58-67c9-48d4-bedb-ed1bb11f70e3 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:24.119947 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 906/1812] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:24 2022] PUT /identity/v3/projects/fb504803729b497ea40a80384946d75e/users/ef5d0b8c3bc145919e19b8a44da4e6d9/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:24.121471 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339e952-a003-463f-a9b0-592742654aa2 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:24.121772 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339e952-a003-463f-a9b0-592742654aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:24.122235 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339e952-a003-463f-a9b0-592742654aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:24.153195 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-d339e952-a003-463f-a9b0-592742654aa2 None None] MFA Rules not processed for user `ef5d0b8c3bc145919e19b8a44da4e6d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:24.154772 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d339e952-a003-463f-a9b0-592742654aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:24.237739 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:24.299310 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 907/1813] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:24 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:38:24.318190 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db387a58-67c9-48d4-bedb-ed1bb11f70e3 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:24.320835 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-db387a58-67c9-48d4-bedb-ed1bb11f70e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:24.464108 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db387a58-67c9-48d4-bedb-ed1bb11f70e3 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:24.469417 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44086fdc-3e65-445b-be8f-bcd96506ebce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:24.470282 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44086fdc-3e65-445b-be8f-bcd96506ebce service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:24.470792 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44086fdc-3e65-445b-be8f-bcd96506ebce service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:24.471077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44086fdc-3e65-445b-be8f-bcd96506ebce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:24.522825 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44086fdc-3e65-445b-be8f-bcd96506ebce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:24.525363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44086fdc-3e65-445b-be8f-bcd96506ebce service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:24.575902 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 907/1814] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:24 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:38:24.745376 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339e952-a003-463f-a9b0-592742654aa2 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:24.748056 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d339e952-a003-463f-a9b0-592742654aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:24.762636 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339e952-a003-463f-a9b0-592742654aa2 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:24.767457 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3101119f-107c-4252-a958-32a47ce3bf6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:24.768201 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101119f-107c-4252-a958-32a47ce3bf6b service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:24.768992 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101119f-107c-4252-a958-32a47ce3bf6b service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:24.769522 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101119f-107c-4252-a958-32a47ce3bf6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:24.772134 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3101119f-107c-4252-a958-32a47ce3bf6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:24.858410 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3101119f-107c-4252-a958-32a47ce3bf6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:24.860277 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3101119f-107c-4252-a958-32a47ce3bf6b service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:24.879048 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 908/1815] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:38:25.691751 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44086fdc-3e65-445b-be8f-bcd96506ebce service cinder] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:25.694215 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-44086fdc-3e65-445b-be8f-bcd96506ebce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:25.761149 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60df1af-c252-4f76-80ab-a03b3dcc2cd0 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:25.762335 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60df1af-c252-4f76-80ab-a03b3dcc2cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:25.763061 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60df1af-c252-4f76-80ab-a03b3dcc2cd0 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:25.767502 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 909/1816] 10.222.0.15 () {54 vars in 921 bytes} [Wed Jun 15 19:38:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:38:25.778788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147c94c-9ec1-49ee-8edb-5fd29ce09e95 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:25.779295 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147c94c-9ec1-49ee-8edb-5fd29ce09e95 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:25.780320 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147c94c-9ec1-49ee-8edb-5fd29ce09e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:25.823326 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-d147c94c-9ec1-49ee-8edb-5fd29ce09e95 None None] MFA Rules not processed for user `3e74ac44e5814122a3226ac023f3688a`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:25.825427 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d147c94c-9ec1-49ee-8edb-5fd29ce09e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:25.882209 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44086fdc-3e65-445b-be8f-bcd96506ebce service cinder] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:25.892821 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c268c193-b1b9-4643-870a-43a1167386fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:25.893891 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c268c193-b1b9-4643-870a-43a1167386fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:25.894345 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c268c193-b1b9-4643-870a-43a1167386fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:25.895093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c268c193-b1b9-4643-870a-43a1167386fd admin admin] PATH_INFO: `/v3/projects/f2df8a0d89a248e6bb4ecb06a1849c05` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:25.900288 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c268c193-b1b9-4643-870a-43a1167386fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2df8a0d89a248e6bb4ecb06a1849c05)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:25.903185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c268c193-b1b9-4643-870a-43a1167386fd admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:25.948025 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 910/1817] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Jun 15 19:38:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:25.962151 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d147c94c-9ec1-49ee-8edb-5fd29ce09e95 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:25.967414 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d147c94c-9ec1-49ee-8edb-5fd29ce09e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:25.988211 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d147c94c-9ec1-49ee-8edb-5fd29ce09e95 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:26.005742 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cadc3a6-d7fb-4675-82af-f1e020dfd5ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:26.007128 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cadc3a6-d7fb-4675-82af-f1e020dfd5ec service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:26.007225 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cadc3a6-d7fb-4675-82af-f1e020dfd5ec service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:26.007552 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cadc3a6-d7fb-4675-82af-f1e020dfd5ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:26.028980 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cadc3a6-d7fb-4675-82af-f1e020dfd5ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:26.031785 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cadc3a6-d7fb-4675-82af-f1e020dfd5ec service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:26.034380 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:26.055580 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-c268c193-b1b9-4643-870a-43a1167386fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2df8a0d89a248e6bb4ecb06a1849c05', 'request_id': 'req-c268c193-b1b9-4643-870a-43a1167386fd'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:26.061929 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 911/1818] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:38:26.072571 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-c268c193-b1b9-4643-870a-43a1167386fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2df8a0d89a248e6bb4ecb06a1849c05', 'request_id': 'req-c268c193-b1b9-4643-870a-43a1167386fd'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:26.081307 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-c268c193-b1b9-4643-870a-43a1167386fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2df8a0d89a248e6bb4ecb06a1849c05', 'request_id': 'req-c268c193-b1b9-4643-870a-43a1167386fd'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:26.097437 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-c268c193-b1b9-4643-870a-43a1167386fd admin admin] The token cache is being invalidate because project f2df8a0d89a248e6bb4ecb06a1849c05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:26.098132 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-c268c193-b1b9-4643-870a-43a1167386fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:26.098932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-c268c193-b1b9-4643-870a-43a1167386fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:26.100987 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 908/1819] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:25 2022] DELETE /identity/v3/projects/f2df8a0d89a248e6bb4ecb06a1849c05 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:26.107172 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d6a1d8-831b-47f0-85b7-79d8a78f93ec None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:26.107462 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d6a1d8-831b-47f0-85b7-79d8a78f93ec None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:26.107960 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d6a1d8-831b-47f0-85b7-79d8a78f93ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:26.148450 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-e9d6a1d8-831b-47f0-85b7-79d8a78f93ec None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:26.150365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e9d6a1d8-831b-47f0-85b7-79d8a78f93ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:26.258954 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:26.313032 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 912/1820] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:26.322422 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ad79c-03d9-43d7-be8d-7da3775e57e7 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:26.322737 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ad79c-03d9-43d7-be8d-7da3775e57e7 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:26.323172 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ad79c-03d9-43d7-be8d-7da3775e57e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:26.359027 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-5f5ad79c-03d9-43d7-be8d-7da3775e57e7 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:26.365204 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-5f5ad79c-03d9-43d7-be8d-7da3775e57e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:26.389133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d6a1d8-831b-47f0-85b7-79d8a78f93ec None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:26.400293 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 909/1821] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:26.402118 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d6a1d8-831b-47f0-85b7-79d8a78f93ec None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:26.413962 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92bdf8d-292c-4605-bcf7-619248373fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:26.414739 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92bdf8d-292c-4605-bcf7-619248373fff admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:26.415076 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92bdf8d-292c-4605-bcf7-619248373fff admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:26.415361 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92bdf8d-292c-4605-bcf7-619248373fff admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:26.416112 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92bdf8d-292c-4605-bcf7-619248373fff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:38:26.416423 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92bdf8d-292c-4605-bcf7-619248373fff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:26.422853 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92bdf8d-292c-4605-bcf7-619248373fff admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:26.454413 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5ad79c-03d9-43d7-be8d-7da3775e57e7 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:26.465939 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 913/1822] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:38:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:38:26.473423 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5ad79c-03d9-43d7-be8d-7da3775e57e7 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:26.491119 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e71485-22ea-471e-bba6-48aed4e8784e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:26.491815 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e71485-22ea-471e-bba6-48aed4e8784e admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:26.494950 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e71485-22ea-471e-bba6-48aed4e8784e admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:26.495324 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e71485-22ea-471e-bba6-48aed4e8784e admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:26.496012 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e71485-22ea-471e-bba6-48aed4e8784e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:26.498321 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e71485-22ea-471e-bba6-48aed4e8784e admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:26.550234 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92bdf8d-292c-4605-bcf7-619248373fff admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:26.559828 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 910/1823] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:26 2022] POST /identity/v3/projects => generated 387 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:38:26.564072 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92bdf8d-292c-4605-bcf7-619248373fff admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:26.747543 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1704069-e2db-4a5e-a075-5f26ea43eb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:26.748243 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1704069-e2db-4a5e-a075-5f26ea43eb86 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:26.748529 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1704069-e2db-4a5e-a075-5f26ea43eb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:26.748810 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1704069-e2db-4a5e-a075-5f26ea43eb86 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:26.749517 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1704069-e2db-4a5e-a075-5f26ea43eb86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:26.751758 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1704069-e2db-4a5e-a075-5f26ea43eb86 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:26.802661 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 914/1824] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:26 2022] POST /identity/v3/users => generated 294 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:26.804607 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e71485-22ea-471e-bba6-48aed4e8784e admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:26.820761 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e71485-22ea-471e-bba6-48aed4e8784e admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:26.834977 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a50e05d-86f4-4bb2-854e-e71aa94356ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:26.835679 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a50e05d-86f4-4bb2-854e-e71aa94356ad admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:26.835970 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a50e05d-86f4-4bb2-854e-e71aa94356ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:26.836258 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a50e05d-86f4-4bb2-854e-e71aa94356ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:26.837025 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a50e05d-86f4-4bb2-854e-e71aa94356ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:26.844562 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a50e05d-86f4-4bb2-854e-e71aa94356ad admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:26.865784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1704069-e2db-4a5e-a075-5f26ea43eb86 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:26.873162 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 911/1825] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:26 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:26.877993 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1704069-e2db-4a5e-a075-5f26ea43eb86 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:26.886303 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332573e2-3412-4e52-8a35-a2982e7d973b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:26.887470 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332573e2-3412-4e52-8a35-a2982e7d973b admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:26.887886 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332573e2-3412-4e52-8a35-a2982e7d973b admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:26.888448 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332573e2-3412-4e52-8a35-a2982e7d973b admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:26.889351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332573e2-3412-4e52-8a35-a2982e7d973b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:26.891672 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332573e2-3412-4e52-8a35-a2982e7d973b admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:26.911242 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a50e05d-86f4-4bb2-854e-e71aa94356ad admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:26.922780 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 915/1826] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:26 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:26.926003 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a50e05d-86f4-4bb2-854e-e71aa94356ad admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:26.940197 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8932803-8198-494f-b08d-3c3dc149b7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:26.941128 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8932803-8198-494f-b08d-3c3dc149b7e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:26.941412 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8932803-8198-494f-b08d-3c3dc149b7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:26.941693 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8932803-8198-494f-b08d-3c3dc149b7e6 admin admin] PATH_INFO: `/v3/projects/93a066a167a64858b708010591b14b1c/users/8b7a6ac5b2eb445990db3811dd1c6f1a/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:26.989497 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8932803-8198-494f-b08d-3c3dc149b7e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93a066a167a64858b708010591b14b1c, user_id=8b7a6ac5b2eb445990db3811dd1c6f1a, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:26.997464 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8932803-8198-494f-b08d-3c3dc149b7e6 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:27.036735 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 912/1827] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:26 2022] PUT /identity/v3/projects/93a066a167a64858b708010591b14b1c/users/8b7a6ac5b2eb445990db3811dd1c6f1a/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:27.038573 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192f848-af16-41f7-83eb-c9302a0c9571 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:27.038879 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192f848-af16-41f7-83eb-c9302a0c9571 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:27.039148 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192f848-af16-41f7-83eb-c9302a0c9571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:27.093815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-3192f848-af16-41f7-83eb-c9302a0c9571 None None] MFA Rules not processed for user `8b7a6ac5b2eb445990db3811dd1c6f1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:27.097877 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3192f848-af16-41f7-83eb-c9302a0c9571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:27.201424 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 916/1828] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:27.202581 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8932803-8198-494f-b08d-3c3dc149b7e6 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:27.300124 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8932803-8198-494f-b08d-3c3dc149b7e6 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:27.303582 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d3df3-5317-45a3-a64a-9979e5ddeb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:27.305590 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d3df3-5317-45a3-a64a-9979e5ddeb12 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:27.306081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d3df3-5317-45a3-a64a-9979e5ddeb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:27.306791 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d3df3-5317-45a3-a64a-9979e5ddeb12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:27.307606 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d3df3-5317-45a3-a64a-9979e5ddeb12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:27.310877 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d3df3-5317-45a3-a64a-9979e5ddeb12 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:27.348359 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 913/1829] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:27 2022] POST /identity/v3/projects => generated 387 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:27.354361 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192f848-af16-41f7-83eb-c9302a0c9571 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:27.369901 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192f848-af16-41f7-83eb-c9302a0c9571 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:27.419266 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:27.432672 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76106b95-3a09-43a6-a8af-04edf55c6f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:27.433353 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76106b95-3a09-43a6-a8af-04edf55c6f8c admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:27.433671 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76106b95-3a09-43a6-a8af-04edf55c6f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:27.433997 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76106b95-3a09-43a6-a8af-04edf55c6f8c admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:27.434743 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76106b95-3a09-43a6-a8af-04edf55c6f8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:27.436733 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76106b95-3a09-43a6-a8af-04edf55c6f8c admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:27.458233 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 917/1830] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:27 2022] POST /identity/v3/users => generated 294 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:27.462197 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d3df3-5317-45a3-a64a-9979e5ddeb12 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:27.473932 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d3df3-5317-45a3-a64a-9979e5ddeb12 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:27.479684 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f35300-1d72-432d-9ac9-78a5f9f5c0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:27.480355 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f35300-1d72-432d-9ac9-78a5f9f5c0e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:27.480674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f35300-1d72-432d-9ac9-78a5f9f5c0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:27.480947 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f35300-1d72-432d-9ac9-78a5f9f5c0e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:27.481703 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f35300-1d72-432d-9ac9-78a5f9f5c0e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:27.483323 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f35300-1d72-432d-9ac9-78a5f9f5c0e9 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:27.494129 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 914/1831] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:27.497990 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76106b95-3a09-43a6-a8af-04edf55c6f8c admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:27.508158 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76106b95-3a09-43a6-a8af-04edf55c6f8c admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:27.513743 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a616d88c-840c-40ae-bb21-520eed9b6f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:27.514411 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616d88c-840c-40ae-bb21-520eed9b6f03 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:27.514696 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616d88c-840c-40ae-bb21-520eed9b6f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:27.514968 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616d88c-840c-40ae-bb21-520eed9b6f03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:27.515695 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a616d88c-840c-40ae-bb21-520eed9b6f03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:27.517769 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a616d88c-840c-40ae-bb21-520eed9b6f03 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:27.528200 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 918/1832] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:27.532406 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f35300-1d72-432d-9ac9-78a5f9f5c0e9 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:27.542799 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f35300-1d72-432d-9ac9-78a5f9f5c0e9 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:27.548040 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a14f5-cbb2-40c4-9ae8-8f17a10f807f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:27.549065 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a14f5-cbb2-40c4-9ae8-8f17a10f807f admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:27.549452 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a14f5-cbb2-40c4-9ae8-8f17a10f807f admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:27.549817 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a14f5-cbb2-40c4-9ae8-8f17a10f807f admin admin] PATH_INFO: `/v3/projects/3b40132403f44a719658d58e30f7b464/users/7442a252a8cc4c25a1c889b60167dce2/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:27.566183 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4a14f5-cbb2-40c4-9ae8-8f17a10f807f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b40132403f44a719658d58e30f7b464, user_id=7442a252a8cc4c25a1c889b60167dce2, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:27.568770 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4a14f5-cbb2-40c4-9ae8-8f17a10f807f admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:27.586669 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 915/1833] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:27 2022] PUT /identity/v3/projects/3b40132403f44a719658d58e30f7b464/users/7442a252a8cc4c25a1c889b60167dce2/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:27.598051 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bcacaf-0f69-4702-8bab-2988dcff2b0d None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:27.598434 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bcacaf-0f69-4702-8bab-2988dcff2b0d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:27.598783 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bcacaf-0f69-4702-8bab-2988dcff2b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:27.627185 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-51bcacaf-0f69-4702-8bab-2988dcff2b0d None None] MFA Rules not processed for user `7442a252a8cc4c25a1c889b60167dce2`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:27.628468 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-51bcacaf-0f69-4702-8bab-2988dcff2b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:27.694877 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 919/1834] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:27.699889 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a14f5-cbb2-40c4-9ae8-8f17a10f807f admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:27.712436 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:27.764422 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a14f5-cbb2-40c4-9ae8-8f17a10f807f admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:27.769025 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c389109-1d99-4f17-850e-6e9ee671b09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:27.769679 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c389109-1d99-4f17-850e-6e9ee671b09e admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:27.769967 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c389109-1d99-4f17-850e-6e9ee671b09e admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:27.770276 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c389109-1d99-4f17-850e-6e9ee671b09e admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:27.770967 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c389109-1d99-4f17-850e-6e9ee671b09e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:27.772872 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c389109-1d99-4f17-850e-6e9ee671b09e admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:27.797982 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 916/1835] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:27 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:27.802179 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bcacaf-0f69-4702-8bab-2988dcff2b0d None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:27.813248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bcacaf-0f69-4702-8bab-2988dcff2b0d None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:27.867815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a6099b-8903-40f5-b353-55990757631f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:27.868404 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6099b-8903-40f5-b353-55990757631f admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:27.868694 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6099b-8903-40f5-b353-55990757631f admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:27.868976 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6099b-8903-40f5-b353-55990757631f admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:27.869709 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a6099b-8903-40f5-b353-55990757631f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:27.872221 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a6099b-8903-40f5-b353-55990757631f admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:27.892606 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 920/1836] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:27 2022] POST /identity/v3/users => generated 299 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:27.896875 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c389109-1d99-4f17-850e-6e9ee671b09e admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:27.907484 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c389109-1d99-4f17-850e-6e9ee671b09e admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:27.912863 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9832fe2f-42b7-47e8-9724-e6dfe2794953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:27.913564 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832fe2f-42b7-47e8-9724-e6dfe2794953 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:27.913846 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832fe2f-42b7-47e8-9724-e6dfe2794953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:27.914154 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832fe2f-42b7-47e8-9724-e6dfe2794953 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:27.914891 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9832fe2f-42b7-47e8-9724-e6dfe2794953 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:27.916501 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9832fe2f-42b7-47e8-9724-e6dfe2794953 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:27.927608 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 917/1837] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:38:27.931865 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a6099b-8903-40f5-b353-55990757631f admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:27.945184 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a6099b-8903-40f5-b353-55990757631f admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:27.950918 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b1815-c303-400a-9d2c-4eb5fc37e72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:27.951877 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b1815-c303-400a-9d2c-4eb5fc37e72a admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:27.952165 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b1815-c303-400a-9d2c-4eb5fc37e72a admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:27.952445 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b1815-c303-400a-9d2c-4eb5fc37e72a admin admin] PATH_INFO: `/v3/projects/b504351549004bf29fb94093a6e814a2/users/4545dad1b2ad497c9470c36986901d1e/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:27.967479 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b1815-c303-400a-9d2c-4eb5fc37e72a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b504351549004bf29fb94093a6e814a2, user_id=4545dad1b2ad497c9470c36986901d1e, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:27.970163 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b1815-c303-400a-9d2c-4eb5fc37e72a admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:27.983477 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 921/1838] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:27 2022] PUT /identity/v3/projects/b504351549004bf29fb94093a6e814a2/users/4545dad1b2ad497c9470c36986901d1e/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:27.987433 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9832fe2f-42b7-47e8-9724-e6dfe2794953 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:27.998062 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9832fe2f-42b7-47e8-9724-e6dfe2794953 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:28.053281 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e8173-a5d2-4d4b-92cd-f381f4cca543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:28.053951 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e8173-a5d2-4d4b-92cd-f381f4cca543 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:28.054261 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e8173-a5d2-4d4b-92cd-f381f4cca543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:28.054535 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e8173-a5d2-4d4b-92cd-f381f4cca543 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:28.055258 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4e8173-a5d2-4d4b-92cd-f381f4cca543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:28.056838 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4e8173-a5d2-4d4b-92cd-f381f4cca543 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:28.067235 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 918/1839] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:27 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:28.071514 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b1815-c303-400a-9d2c-4eb5fc37e72a admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:28.082392 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b1815-c303-400a-9d2c-4eb5fc37e72a admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:28.087104 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdcdd32-1ded-48d3-92c7-c9e76ab03f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:28.087993 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcdd32-1ded-48d3-92c7-c9e76ab03f82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:28.088277 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcdd32-1ded-48d3-92c7-c9e76ab03f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:28.088555 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcdd32-1ded-48d3-92c7-c9e76ab03f82 admin admin] PATH_INFO: `/v3/projects/b504351549004bf29fb94093a6e814a2/users/4545dad1b2ad497c9470c36986901d1e/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:28.091249 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdcdd32-1ded-48d3-92c7-c9e76ab03f82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b504351549004bf29fb94093a6e814a2, user_id=4545dad1b2ad497c9470c36986901d1e, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:28.093976 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdcdd32-1ded-48d3-92c7-c9e76ab03f82 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:28.105083 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 922/1840] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:28 2022] PUT /identity/v3/projects/b504351549004bf29fb94093a6e814a2/users/4545dad1b2ad497c9470c36986901d1e/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:38:28.117031 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141253f-b835-4eee-90a1-d8e04dec5ee2 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:28.117404 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141253f-b835-4eee-90a1-d8e04dec5ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:28.117692 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141253f-b835-4eee-90a1-d8e04dec5ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:28.143687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-a141253f-b835-4eee-90a1-d8e04dec5ee2 None None] MFA Rules not processed for user `4545dad1b2ad497c9470c36986901d1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:28.144840 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-a141253f-b835-4eee-90a1-d8e04dec5ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:28.231625 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 919/1841] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:28 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:28.242547 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdcdd32-1ded-48d3-92c7-c9e76ab03f82 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:28.244509 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-efdcdd32-1ded-48d3-92c7-c9e76ab03f82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:28.256951 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:28.308448 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdcdd32-1ded-48d3-92c7-c9e76ab03f82 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:28.312160 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae2024-2e37-43d8-905e-136d3fe7f10b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:28.312808 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae2024-2e37-43d8-905e-136d3fe7f10b service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:28.313092 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae2024-2e37-43d8-905e-136d3fe7f10b service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:28.313368 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae2024-2e37-43d8-905e-136d3fe7f10b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:28.322907 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae2024-2e37-43d8-905e-136d3fe7f10b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:28.324887 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae2024-2e37-43d8-905e-136d3fe7f10b service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:28.337290 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 923/1842] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:28 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:38:28.942372 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a141253f-b835-4eee-90a1-d8e04dec5ee2 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:28.952942 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a141253f-b835-4eee-90a1-d8e04dec5ee2 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:28.956902 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbda309-238a-4618-bdfd-343ae9923351 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:28.958894 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbda309-238a-4618-bdfd-343ae9923351 service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:28.959184 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbda309-238a-4618-bdfd-343ae9923351 service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:28.959618 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbda309-238a-4618-bdfd-343ae9923351 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:28.964330 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:29.024134 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbda309-238a-4618-bdfd-343ae9923351 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:29.025566 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbda309-238a-4618-bdfd-343ae9923351 service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:29.035101 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 920/1843] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:28 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:38:31.112366 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae2024-2e37-43d8-905e-136d3fe7f10b service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:31.113803 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ebae2024-2e37-43d8-905e-136d3fe7f10b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:31.182393 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae2024-2e37-43d8-905e-136d3fe7f10b service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:31.187480 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70682b96-c8c6-4d90-aaa3-5872c7558e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:31.188353 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70682b96-c8c6-4d90-aaa3-5872c7558e61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:31.188635 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70682b96-c8c6-4d90-aaa3-5872c7558e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:31.188913 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70682b96-c8c6-4d90-aaa3-5872c7558e61 admin admin] PATH_INFO: `/v3/users/fc43ce0c9a8f4595919d447f1f9411c2` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:31.190123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70682b96-c8c6-4d90-aaa3-5872c7558e61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc43ce0c9a8f4595919d447f1f9411c2)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:31.192125 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70682b96-c8c6-4d90-aaa3-5872c7558e61 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:31.236857 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-70682b96-c8c6-4d90-aaa3-5872c7558e61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc43ce0c9a8f4595919d447f1f9411c2'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:31.237534 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-70682b96-c8c6-4d90-aaa3-5872c7558e61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc43ce0c9a8f4595919d447f1f9411c2'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:31.238175 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-70682b96-c8c6-4d90-aaa3-5872c7558e61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc43ce0c9a8f4595919d447f1f9411c2'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:31.250740 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-70682b96-c8c6-4d90-aaa3-5872c7558e61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc43ce0c9a8f4595919d447f1f9411c2'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:31.257541 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-70682b96-c8c6-4d90-aaa3-5872c7558e61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc43ce0c9a8f4595919d447f1f9411c2'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:31.270159 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 924/1844] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:38:31 2022] DELETE /identity/v3/users/fc43ce0c9a8f4595919d447f1f9411c2 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:31.274762 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2c989-dfaf-493c-892a-d86ea7484464 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:31.275206 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2c989-dfaf-493c-892a-d86ea7484464 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:31.275349 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2c989-dfaf-493c-892a-d86ea7484464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:31.292253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-06a2c989-dfaf-493c-892a-d86ea7484464 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:31.293944 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-06a2c989-dfaf-493c-892a-d86ea7484464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:31.353931 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 921/1845] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:38:31.364237 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70682b96-c8c6-4d90-aaa3-5872c7558e61 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:31.365843 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-70682b96-c8c6-4d90-aaa3-5872c7558e61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:31.401542 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:31.427925 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70682b96-c8c6-4d90-aaa3-5872c7558e61 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:31.431602 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdbbdc6-cfca-4ac8-9e13-177ce252abe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:31.432253 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdbbdc6-cfca-4ac8-9e13-177ce252abe3 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:31.432606 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdbbdc6-cfca-4ac8-9e13-177ce252abe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:31.432810 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdbbdc6-cfca-4ac8-9e13-177ce252abe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:31.442672 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdbbdc6-cfca-4ac8-9e13-177ce252abe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:31.444274 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdbbdc6-cfca-4ac8-9e13-177ce252abe3 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:31.454847 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 925/1846] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:38:31.756805 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2c989-dfaf-493c-892a-d86ea7484464 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:31.758274 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-06a2c989-dfaf-493c-892a-d86ea7484464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:31.778376 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2c989-dfaf-493c-892a-d86ea7484464 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:31.782983 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77579c7e-f2ab-4e24-9d0d-0a66547dc01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:31.784032 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77579c7e-f2ab-4e24-9d0d-0a66547dc01e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:31.784314 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77579c7e-f2ab-4e24-9d0d-0a66547dc01e admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:31.784590 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77579c7e-f2ab-4e24-9d0d-0a66547dc01e admin admin] PATH_INFO: `/v3/projects/55c213aa313b4afb8bdaedcde44a732b` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:31.785795 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77579c7e-f2ab-4e24-9d0d-0a66547dc01e admin admin] RBAC: Authorizing `identity:delete_project(project_id=55c213aa313b4afb8bdaedcde44a732b)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:31.788060 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77579c7e-f2ab-4e24-9d0d-0a66547dc01e admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:31.854611 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-77579c7e-f2ab-4e24-9d0d-0a66547dc01e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55c213aa313b4afb8bdaedcde44a732b', 'request_id': 'req-77579c7e-f2ab-4e24-9d0d-0a66547dc01e'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:31.865545 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-77579c7e-f2ab-4e24-9d0d-0a66547dc01e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55c213aa313b4afb8bdaedcde44a732b', 'request_id': 'req-77579c7e-f2ab-4e24-9d0d-0a66547dc01e'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:31.876557 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-77579c7e-f2ab-4e24-9d0d-0a66547dc01e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55c213aa313b4afb8bdaedcde44a732b', 'request_id': 'req-77579c7e-f2ab-4e24-9d0d-0a66547dc01e'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:31.892818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-77579c7e-f2ab-4e24-9d0d-0a66547dc01e admin admin] The token cache is being invalidate because project 55c213aa313b4afb8bdaedcde44a732b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:31.893444 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-77579c7e-f2ab-4e24-9d0d-0a66547dc01e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:31.894447 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-77579c7e-f2ab-4e24-9d0d-0a66547dc01e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:31.896634 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 922/1847] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:31 2022] DELETE /identity/v3/projects/55c213aa313b4afb8bdaedcde44a732b => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:31.900973 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdbbdc6-cfca-4ac8-9e13-177ce252abe3 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:31.915551 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdbbdc6-cfca-4ac8-9e13-177ce252abe3 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:32.033392 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33564ad5-e711-4c90-86db-c8da6d45ad78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:32.034569 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33564ad5-e711-4c90-86db-c8da6d45ad78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:32.034849 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33564ad5-e711-4c90-86db-c8da6d45ad78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:32.035120 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33564ad5-e711-4c90-86db-c8da6d45ad78 admin admin] PATH_INFO: `/v3/users/ef5d0b8c3bc145919e19b8a44da4e6d9` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:32.037276 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33564ad5-e711-4c90-86db-c8da6d45ad78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef5d0b8c3bc145919e19b8a44da4e6d9)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:32.045496 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33564ad5-e711-4c90-86db-c8da6d45ad78 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:32.126895 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-33564ad5-e711-4c90-86db-c8da6d45ad78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef5d0b8c3bc145919e19b8a44da4e6d9'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:32.127864 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-33564ad5-e711-4c90-86db-c8da6d45ad78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef5d0b8c3bc145919e19b8a44da4e6d9'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:32.129723 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-33564ad5-e711-4c90-86db-c8da6d45ad78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef5d0b8c3bc145919e19b8a44da4e6d9'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:32.150007 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-33564ad5-e711-4c90-86db-c8da6d45ad78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef5d0b8c3bc145919e19b8a44da4e6d9'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:32.166813 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-33564ad5-e711-4c90-86db-c8da6d45ad78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef5d0b8c3bc145919e19b8a44da4e6d9'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:32.190293 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 926/1848] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:38:31 2022] DELETE /identity/v3/users/ef5d0b8c3bc145919e19b8a44da4e6d9 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:38:33.016099 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77579c7e-f2ab-4e24-9d0d-0a66547dc01e admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:33.017674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-77579c7e-f2ab-4e24-9d0d-0a66547dc01e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:33.035269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77579c7e-f2ab-4e24-9d0d-0a66547dc01e admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:33.062037 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:33.095767 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e3250d-3890-4eb0-a026-573367c8251c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:33.096668 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e3250d-3890-4eb0-a026-573367c8251c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:33.096954 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e3250d-3890-4eb0-a026-573367c8251c admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:33.097232 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e3250d-3890-4eb0-a026-573367c8251c admin admin] PATH_INFO: `/v3/projects/fb504803729b497ea40a80384946d75e` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:33.098499 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e3250d-3890-4eb0-a026-573367c8251c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb504803729b497ea40a80384946d75e)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:33.100438 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e3250d-3890-4eb0-a026-573367c8251c admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:33.152756 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-48e3250d-3890-4eb0-a026-573367c8251c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb504803729b497ea40a80384946d75e', 'request_id': 'req-48e3250d-3890-4eb0-a026-573367c8251c'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:33.161538 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-48e3250d-3890-4eb0-a026-573367c8251c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb504803729b497ea40a80384946d75e', 'request_id': 'req-48e3250d-3890-4eb0-a026-573367c8251c'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:33.170655 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-48e3250d-3890-4eb0-a026-573367c8251c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb504803729b497ea40a80384946d75e', 'request_id': 'req-48e3250d-3890-4eb0-a026-573367c8251c'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:33.181071 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-48e3250d-3890-4eb0-a026-573367c8251c admin admin] The token cache is being invalidate because project fb504803729b497ea40a80384946d75e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:33.181363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-48e3250d-3890-4eb0-a026-573367c8251c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:33.182123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-48e3250d-3890-4eb0-a026-573367c8251c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:33.184734 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 923/1849] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:33 2022] DELETE /identity/v3/projects/fb504803729b497ea40a80384946d75e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:33.189389 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36cded-6e30-4034-a122-9d1970bc901c None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:33.189823 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36cded-6e30-4034-a122-9d1970bc901c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:33.190167 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36cded-6e30-4034-a122-9d1970bc901c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:33.207757 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-0e36cded-6e30-4034-a122-9d1970bc901c None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:33.209222 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0e36cded-6e30-4034-a122-9d1970bc901c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:33.263066 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:33.280356 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 927/1850] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:38:33.290235 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291dc9c-e695-43ce-8f8f-dbb6a967b375 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:33.290235 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291dc9c-e695-43ce-8f8f-dbb6a967b375 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:33.290235 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291dc9c-e695-43ce-8f8f-dbb6a967b375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:33.299828 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e36cded-6e30-4034-a122-9d1970bc901c None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:33.301242 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0e36cded-6e30-4034-a122-9d1970bc901c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:33.336713 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-6291dc9c-e695-43ce-8f8f-dbb6a967b375 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:33.338626 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-6291dc9c-e695-43ce-8f8f-dbb6a967b375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:33.349739 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 924/1851] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:33.355102 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6291dc9c-e695-43ce-8f8f-dbb6a967b375 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:33.365515 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6291dc9c-e695-43ce-8f8f-dbb6a967b375 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:33.370412 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f3299-2c5e-4309-a85b-ae6e8ac2c005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:33.371107 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f3299-2c5e-4309-a85b-ae6e8ac2c005 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:33.371394 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f3299-2c5e-4309-a85b-ae6e8ac2c005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:33.371677 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f3299-2c5e-4309-a85b-ae6e8ac2c005 admin admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:33.372435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f3299-2c5e-4309-a85b-ae6e8ac2c005 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:38:33.372748 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f3299-2c5e-4309-a85b-ae6e8ac2c005 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:33.374370 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f3299-2c5e-4309-a85b-ae6e8ac2c005 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:33.376319 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e36cded-6e30-4034-a122-9d1970bc901c None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:33.380604 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959f65d-bfb4-4721-80f5-96f27cc1c4ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:33.381449 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959f65d-bfb4-4721-80f5-96f27cc1c4ab service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:33.382449 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959f65d-bfb4-4721-80f5-96f27cc1c4ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:33.382907 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959f65d-bfb4-4721-80f5-96f27cc1c4ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:33.386436 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8959f65d-bfb4-4721-80f5-96f27cc1c4ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:33.388878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 925/1852] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:38:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:38:33.397650 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f3299-2c5e-4309-a85b-ae6e8ac2c005 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:33.410335 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f3299-2c5e-4309-a85b-ae6e8ac2c005 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:33.425363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c74d6-7370-475a-a25e-942ae829e426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:33.426216 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c74d6-7370-475a-a25e-942ae829e426 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:33.426541 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c74d6-7370-475a-a25e-942ae829e426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:33.427061 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c74d6-7370-475a-a25e-942ae829e426 admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:33.427821 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266c74d6-7370-475a-a25e-942ae829e426 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:33.429797 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266c74d6-7370-475a-a25e-942ae829e426 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:33.465701 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 926/1853] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:33 2022] POST /identity/v3/projects => generated 365 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:33.469147 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c74d6-7370-475a-a25e-942ae829e426 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:33.491879 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8959f65d-bfb4-4721-80f5-96f27cc1c4ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:33.494417 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8959f65d-bfb4-4721-80f5-96f27cc1c4ab service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:33.509818 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 928/1854] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:38:33.558064 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c74d6-7370-475a-a25e-942ae829e426 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:33.565280 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236af9ea-bb7c-426b-85be-51229af78a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:33.566415 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236af9ea-bb7c-426b-85be-51229af78a8c admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:33.566736 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236af9ea-bb7c-426b-85be-51229af78a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:33.567018 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236af9ea-bb7c-426b-85be-51229af78a8c admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:33.567859 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236af9ea-bb7c-426b-85be-51229af78a8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:33.570268 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236af9ea-bb7c-426b-85be-51229af78a8c admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:33.597683 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 927/1855] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:33 2022] POST /identity/v3/users => generated 283 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:33.602945 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959f65d-bfb4-4721-80f5-96f27cc1c4ab service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:33.616932 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959f65d-bfb4-4721-80f5-96f27cc1c4ab service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:33.632090 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec30f12-255e-468d-a104-258bc4478e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:33.632090 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec30f12-255e-468d-a104-258bc4478e59 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:33.632090 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec30f12-255e-468d-a104-258bc4478e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:33.632090 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec30f12-255e-468d-a104-258bc4478e59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:33.632373 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec30f12-255e-468d-a104-258bc4478e59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:33.635241 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec30f12-255e-468d-a104-258bc4478e59 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:33.679571 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 929/1856] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:33.686346 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236af9ea-bb7c-426b-85be-51229af78a8c admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:33.704071 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236af9ea-bb7c-426b-85be-51229af78a8c admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:33.717033 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9667cc-59c6-4342-a532-1dd572d017f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:33.717729 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9667cc-59c6-4342-a532-1dd572d017f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:33.718013 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9667cc-59c6-4342-a532-1dd572d017f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:33.718322 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9667cc-59c6-4342-a532-1dd572d017f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:33.719063 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9667cc-59c6-4342-a532-1dd572d017f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:33.723356 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9667cc-59c6-4342-a532-1dd572d017f9 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:33.731368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:33.753892 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 928/1857] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:33.755128 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec30f12-255e-468d-a104-258bc4478e59 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:33.768721 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec30f12-255e-468d-a104-258bc4478e59 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:33.782632 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18dc68-e41f-438b-bc82-ef8bdb98897a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:33.783286 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18dc68-e41f-438b-bc82-ef8bdb98897a admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:33.783769 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18dc68-e41f-438b-bc82-ef8bdb98897a admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:33.784109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18dc68-e41f-438b-bc82-ef8bdb98897a admin admin] PATH_INFO: `/v3/projects/a352916bad2f4935beb3bd9979bfd61c/users/56e6903cec2640cb8da3d62c45116820/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:33.826462 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d18dc68-e41f-438b-bc82-ef8bdb98897a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a352916bad2f4935beb3bd9979bfd61c, user_id=56e6903cec2640cb8da3d62c45116820, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:33.832275 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d18dc68-e41f-438b-bc82-ef8bdb98897a admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:33.877457 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75322b-1449-447f-bdd2-cee21f125d0d None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:33.877770 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75322b-1449-447f-bdd2-cee21f125d0d None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:33.878028 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75322b-1449-447f-bdd2-cee21f125d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:33.891122 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 930/1858] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:33 2022] PUT /identity/v3/projects/a352916bad2f4935beb3bd9979bfd61c/users/56e6903cec2640cb8da3d62c45116820/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:33.943194 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-7f75322b-1449-447f-bdd2-cee21f125d0d None None] MFA Rules not processed for user `56e6903cec2640cb8da3d62c45116820`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:33.945443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-7f75322b-1449-447f-bdd2-cee21f125d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:34.110696 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 929/1859] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:33 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:38:34.115126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18dc68-e41f-438b-bc82-ef8bdb98897a admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:34.265270 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18dc68-e41f-438b-bc82-ef8bdb98897a admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:34.267592 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f75322b-1449-447f-bdd2-cee21f125d0d None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:34.269012 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-7f75322b-1449-447f-bdd2-cee21f125d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:34.275748 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721acd9c-5119-4e55-b04a-bdeaa9e8d193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:34.277364 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721acd9c-5119-4e55-b04a-bdeaa9e8d193 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:34.278581 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721acd9c-5119-4e55-b04a-bdeaa9e8d193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:34.279106 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721acd9c-5119-4e55-b04a-bdeaa9e8d193 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:34.282003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721acd9c-5119-4e55-b04a-bdeaa9e8d193 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:34.288088 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721acd9c-5119-4e55-b04a-bdeaa9e8d193 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:34.304204 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f75322b-1449-447f-bdd2-cee21f125d0d None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:34.318561 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfee392-0ddc-4370-b6c3-ecebad4dde98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:34.321604 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfee392-0ddc-4370-b6c3-ecebad4dde98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:34.321979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfee392-0ddc-4370-b6c3-ecebad4dde98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:34.322299 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfee392-0ddc-4370-b6c3-ecebad4dde98 admin admin] PATH_INFO: `/v3/users/87653066617d49bab8dc39e8cafd0e7b` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:34.324849 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfee392-0ddc-4370-b6c3-ecebad4dde98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87653066617d49bab8dc39e8cafd0e7b)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:34.328198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfee392-0ddc-4370-b6c3-ecebad4dde98 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:34.340784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:34.360857 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 931/1860] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:34 2022] POST /identity/v3/projects => generated 365 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:34.361743 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721acd9c-5119-4e55-b04a-bdeaa9e8d193 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:34.444382 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-fdfee392-0ddc-4370-b6c3-ecebad4dde98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87653066617d49bab8dc39e8cafd0e7b'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:34.445038 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-fdfee392-0ddc-4370-b6c3-ecebad4dde98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87653066617d49bab8dc39e8cafd0e7b'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:34.462075 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-fdfee392-0ddc-4370-b6c3-ecebad4dde98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87653066617d49bab8dc39e8cafd0e7b'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:34.463436 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-fdfee392-0ddc-4370-b6c3-ecebad4dde98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87653066617d49bab8dc39e8cafd0e7b'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:34.469988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721acd9c-5119-4e55-b04a-bdeaa9e8d193 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:34.476304 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa44b76d-4a29-474c-960c-5398df6fe9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:34.477294 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44b76d-4a29-474c-960c-5398df6fe9ed admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:34.477689 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44b76d-4a29-474c-960c-5398df6fe9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:34.478027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44b76d-4a29-474c-960c-5398df6fe9ed admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:34.478819 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa44b76d-4a29-474c-960c-5398df6fe9ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:34.481726 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa44b76d-4a29-474c-960c-5398df6fe9ed admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:34.483959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-fdfee392-0ddc-4370-b6c3-ecebad4dde98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87653066617d49bab8dc39e8cafd0e7b'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:34.513850 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 932/1861] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:34 2022] POST /identity/v3/users => generated 289 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:34.519062 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa44b76d-4a29-474c-960c-5398df6fe9ed admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:34.520672 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-aa44b76d-4a29-474c-960c-5398df6fe9ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:34.528208 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 930/1862] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:38:34 2022] DELETE /identity/v3/users/87653066617d49bab8dc39e8cafd0e7b => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:34.532475 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c44292-e985-4203-a3d0-3af603d4b893 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:34.532984 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c44292-e985-4203-a3d0-3af603d4b893 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:34.534079 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c44292-e985-4203-a3d0-3af603d4b893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:34.541238 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa44b76d-4a29-474c-960c-5398df6fe9ed admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:34.567244 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-98c44292-e985-4203-a3d0-3af603d4b893 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:34.568710 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-98c44292-e985-4203-a3d0-3af603d4b893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:34.653749 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:34.691373 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 931/1863] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:34.699155 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673390a4-67c0-4183-8d37-b89f9b578b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:34.699933 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673390a4-67c0-4183-8d37-b89f9b578b57 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:34.700312 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673390a4-67c0-4183-8d37-b89f9b578b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:34.700665 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673390a4-67c0-4183-8d37-b89f9b578b57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:34.700994 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c44292-e985-4203-a3d0-3af603d4b893 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:34.701538 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673390a4-67c0-4183-8d37-b89f9b578b57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:34.703137 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-98c44292-e985-4203-a3d0-3af603d4b893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:34.703383 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673390a4-67c0-4183-8d37-b89f9b578b57 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:34.716135 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 933/1864] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:34 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:34.720554 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673390a4-67c0-4183-8d37-b89f9b578b57 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:34.738905 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673390a4-67c0-4183-8d37-b89f9b578b57 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:34.746746 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5fbbc-ad66-46cc-875c-e2cc1e2e1789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:34.747808 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5fbbc-ad66-46cc-875c-e2cc1e2e1789 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:34.748096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5fbbc-ad66-46cc-875c-e2cc1e2e1789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:34.748373 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5fbbc-ad66-46cc-875c-e2cc1e2e1789 admin admin] PATH_INFO: `/v3/projects/39337f1f97074cff8beaeec245d62e91/users/3dcb6816aef145409e264bbb5f526fc2/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:34.782223 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5fbbc-ad66-46cc-875c-e2cc1e2e1789 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39337f1f97074cff8beaeec245d62e91, user_id=3dcb6816aef145409e264bbb5f526fc2, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:34.786659 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5fbbc-ad66-46cc-875c-e2cc1e2e1789 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:34.787590 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c44292-e985-4203-a3d0-3af603d4b893 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:34.793815 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a017241-2df5-4f2d-aaa2-f2f5f70ae3a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:34.794736 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a017241-2df5-4f2d-aaa2-f2f5f70ae3a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:34.795229 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a017241-2df5-4f2d-aaa2-f2f5f70ae3a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:34.795506 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a017241-2df5-4f2d-aaa2-f2f5f70ae3a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:34.806378 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 934/1865] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:34 2022] PUT /identity/v3/projects/39337f1f97074cff8beaeec245d62e91/users/3dcb6816aef145409e264bbb5f526fc2/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:34.807613 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a017241-2df5-4f2d-aaa2-f2f5f70ae3a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:34.809200 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a017241-2df5-4f2d-aaa2-f2f5f70ae3a6 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:34.810561 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5fbbc-ad66-46cc-875c-e2cc1e2e1789 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:34.826784 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 932/1866] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:38:34.899888 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5fbbc-ad66-46cc-875c-e2cc1e2e1789 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:34.905919 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa6b78-f619-4bf8-8c6b-ce687b5daf35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:34.906920 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa6b78-f619-4bf8-8c6b-ce687b5daf35 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:34.907210 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa6b78-f619-4bf8-8c6b-ce687b5daf35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:34.907527 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa6b78-f619-4bf8-8c6b-ce687b5daf35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:34.908540 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa6b78-f619-4bf8-8c6b-ce687b5daf35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:34.910662 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa6b78-f619-4bf8-8c6b-ce687b5daf35 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:34.913018 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:34.933900 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 935/1867] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:34 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:34.934459 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a017241-2df5-4f2d-aaa2-f2f5f70ae3a6 service neutron] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:34.948089 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a017241-2df5-4f2d-aaa2-f2f5f70ae3a6 service neutron] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:34.970222 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d914cb6e-a38b-4762-8f1c-0cbc75ea5acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:34.971505 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914cb6e-a38b-4762-8f1c-0cbc75ea5acd admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:34.971795 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914cb6e-a38b-4762-8f1c-0cbc75ea5acd admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:34.972224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914cb6e-a38b-4762-8f1c-0cbc75ea5acd admin admin] PATH_INFO: `/v3/projects/39337f1f97074cff8beaeec245d62e91/users/3dcb6816aef145409e264bbb5f526fc2/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:34.975076 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d914cb6e-a38b-4762-8f1c-0cbc75ea5acd admin admin] RBAC: Authorizing `identity:create_grant(project_id=39337f1f97074cff8beaeec245d62e91, user_id=3dcb6816aef145409e264bbb5f526fc2, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:34.978931 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d914cb6e-a38b-4762-8f1c-0cbc75ea5acd admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:35.006959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 933/1868] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:34 2022] PUT /identity/v3/projects/39337f1f97074cff8beaeec245d62e91/users/3dcb6816aef145409e264bbb5f526fc2/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:35.018431 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7327e-94eb-45f0-914b-0099dbf7a8fd None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:35.018431 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7327e-94eb-45f0-914b-0099dbf7a8fd None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:35.019007 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7327e-94eb-45f0-914b-0099dbf7a8fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:35.053802 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-bfd7327e-94eb-45f0-914b-0099dbf7a8fd None None] MFA Rules not processed for user `3dcb6816aef145409e264bbb5f526fc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:35.055527 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-bfd7327e-94eb-45f0-914b-0099dbf7a8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:35.164168 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 936/1869] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:35 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:35.176219 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d914cb6e-a38b-4762-8f1c-0cbc75ea5acd admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:35.177957 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-d914cb6e-a38b-4762-8f1c-0cbc75ea5acd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:35.254523 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d914cb6e-a38b-4762-8f1c-0cbc75ea5acd admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:35.260594 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd1311-d9be-491b-bf6d-a08170dbe4fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:35.261200 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd1311-d9be-491b-bf6d-a08170dbe4fd service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:35.262013 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd1311-d9be-491b-bf6d-a08170dbe4fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:35.262489 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd1311-d9be-491b-bf6d-a08170dbe4fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:35.272018 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd1311-d9be-491b-bf6d-a08170dbe4fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:35.273448 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd1311-d9be-491b-bf6d-a08170dbe4fd service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:35.284292 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 934/1870] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:35 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:38:35.350325 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7327e-94eb-45f0-914b-0099dbf7a8fd None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:35.351726 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-bfd7327e-94eb-45f0-914b-0099dbf7a8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:35.374611 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd1311-d9be-491b-bf6d-a08170dbe4fd service cinder] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:35.384637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd1311-d9be-491b-bf6d-a08170dbe4fd service cinder] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:35.389110 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c024e0e9-70bd-4def-b412-d67c30119f0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:35.389758 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c024e0e9-70bd-4def-b412-d67c30119f0c service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:35.390045 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c024e0e9-70bd-4def-b412-d67c30119f0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:35.390571 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c024e0e9-70bd-4def-b412-d67c30119f0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:35.392578 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-c024e0e9-70bd-4def-b412-d67c30119f0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:35.496847 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c024e0e9-70bd-4def-b412-d67c30119f0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:35.498589 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c024e0e9-70bd-4def-b412-d67c30119f0c service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:35.507933 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7327e-94eb-45f0-914b-0099dbf7a8fd None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:35.513480 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55050038-2b60-4461-b234-36c367989d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:35.514508 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55050038-2b60-4461-b234-36c367989d73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:35.514793 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55050038-2b60-4461-b234-36c367989d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:35.515068 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55050038-2b60-4461-b234-36c367989d73 admin admin] PATH_INFO: `/v3/projects/3824390d416742efab487d3d676f468c` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:35.516287 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55050038-2b60-4461-b234-36c367989d73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3824390d416742efab487d3d676f468c)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:35.518681 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55050038-2b60-4461-b234-36c367989d73 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:35.521502 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 935/1871] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:35 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:38:35.607191 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-55050038-2b60-4461-b234-36c367989d73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3824390d416742efab487d3d676f468c', 'request_id': 'req-55050038-2b60-4461-b234-36c367989d73'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:35.634041 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-55050038-2b60-4461-b234-36c367989d73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3824390d416742efab487d3d676f468c', 'request_id': 'req-55050038-2b60-4461-b234-36c367989d73'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:35.651599 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-55050038-2b60-4461-b234-36c367989d73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3824390d416742efab487d3d676f468c', 'request_id': 'req-55050038-2b60-4461-b234-36c367989d73'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:35.667499 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-55050038-2b60-4461-b234-36c367989d73 admin admin] The token cache is being invalidate because project 3824390d416742efab487d3d676f468c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:35.668669 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-55050038-2b60-4461-b234-36c367989d73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:35.669392 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-55050038-2b60-4461-b234-36c367989d73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:35.671923 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 937/1872] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:35 2022] DELETE /identity/v3/projects/3824390d416742efab487d3d676f468c => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:35.679211 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c024e0e9-70bd-4def-b412-d67c30119f0c service cinder] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:35.691265 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c024e0e9-70bd-4def-b412-d67c30119f0c service cinder] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:35.723121 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:35.778751 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d7565-8b77-45e9-bb4c-599f4c0e5311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:35.780247 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d7565-8b77-45e9-bb4c-599f4c0e5311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:35.780602 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d7565-8b77-45e9-bb4c-599f4c0e5311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:35.780890 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d7565-8b77-45e9-bb4c-599f4c0e5311 admin admin] PATH_INFO: `/v3/users/069b950d675343d3a0e8dc393f49d3c8` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:35.782593 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496d7565-8b77-45e9-bb4c-599f4c0e5311 admin admin] RBAC: Authorizing `identity:delete_user(user_id=069b950d675343d3a0e8dc393f49d3c8)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:35.786058 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496d7565-8b77-45e9-bb4c-599f4c0e5311 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:35.865067 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-496d7565-8b77-45e9-bb4c-599f4c0e5311 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '069b950d675343d3a0e8dc393f49d3c8'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:35.866509 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-496d7565-8b77-45e9-bb4c-599f4c0e5311 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '069b950d675343d3a0e8dc393f49d3c8'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:35.876997 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-496d7565-8b77-45e9-bb4c-599f4c0e5311 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '069b950d675343d3a0e8dc393f49d3c8'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:35.877778 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-496d7565-8b77-45e9-bb4c-599f4c0e5311 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '069b950d675343d3a0e8dc393f49d3c8'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:35.887209 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-496d7565-8b77-45e9-bb4c-599f4c0e5311 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '069b950d675343d3a0e8dc393f49d3c8'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:35.908395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 936/1873] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:38:35 2022] DELETE /identity/v3/users/069b950d675343d3a0e8dc393f49d3c8 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:36.202799 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55050038-2b60-4461-b234-36c367989d73 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:36.205106 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-55050038-2b60-4461-b234-36c367989d73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:36.243893 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55050038-2b60-4461-b234-36c367989d73 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:36.254682 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:36.319420 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e76347-3446-4e64-a5db-efca0776cfe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:36.320428 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e76347-3446-4e64-a5db-efca0776cfe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:36.320710 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e76347-3446-4e64-a5db-efca0776cfe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:36.320985 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e76347-3446-4e64-a5db-efca0776cfe2 admin admin] PATH_INFO: `/v3/projects/736cecd0d293468db2803ea4c1aa15e8` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:36.322819 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e76347-3446-4e64-a5db-efca0776cfe2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=736cecd0d293468db2803ea4c1aa15e8)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:36.325087 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e76347-3446-4e64-a5db-efca0776cfe2 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:36.393496 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-55e76347-3446-4e64-a5db-efca0776cfe2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '736cecd0d293468db2803ea4c1aa15e8', 'request_id': 'req-55e76347-3446-4e64-a5db-efca0776cfe2'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:36.412832 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-55e76347-3446-4e64-a5db-efca0776cfe2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '736cecd0d293468db2803ea4c1aa15e8', 'request_id': 'req-55e76347-3446-4e64-a5db-efca0776cfe2'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:36.420846 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-55e76347-3446-4e64-a5db-efca0776cfe2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '736cecd0d293468db2803ea4c1aa15e8', 'request_id': 'req-55e76347-3446-4e64-a5db-efca0776cfe2'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:36.431864 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-55e76347-3446-4e64-a5db-efca0776cfe2 admin admin] The token cache is being invalidate because project 736cecd0d293468db2803ea4c1aa15e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:36.432178 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-55e76347-3446-4e64-a5db-efca0776cfe2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:36.433068 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-55e76347-3446-4e64-a5db-efca0776cfe2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:36.434892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 938/1874] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:36 2022] DELETE /identity/v3/projects/736cecd0d293468db2803ea4c1aa15e8 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:36.439695 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbea89c-cfb7-4883-8388-11a64e6a38a5 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:36.440155 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbea89c-cfb7-4883-8388-11a64e6a38a5 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:36.440275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbea89c-cfb7-4883-8388-11a64e6a38a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:36.458788 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-fdbea89c-cfb7-4883-8388-11a64e6a38a5 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:36.459978 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-fdbea89c-cfb7-4883-8388-11a64e6a38a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:36.527619 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 937/1875] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:36.532475 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e76347-3446-4e64-a5db-efca0776cfe2 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:36.533882 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-55e76347-3446-4e64-a5db-efca0776cfe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:36.539624 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f14c8e8-478e-4af2-b47d-535a29891c80 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:36.539920 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f14c8e8-478e-4af2-b47d-535a29891c80 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:36.540322 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f14c8e8-478e-4af2-b47d-535a29891c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:36.547213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e76347-3446-4e64-a5db-efca0776cfe2 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:36.551923 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296f2f0-5204-4ec9-b569-3da050dfffa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:36.552819 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296f2f0-5204-4ec9-b569-3da050dfffa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:36.553116 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296f2f0-5204-4ec9-b569-3da050dfffa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:36.553598 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296f2f0-5204-4ec9-b569-3da050dfffa1 admin admin] PATH_INFO: `/v3/users/8b7a6ac5b2eb445990db3811dd1c6f1a` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:36.554588 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b296f2f0-5204-4ec9-b569-3da050dfffa1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b7a6ac5b2eb445990db3811dd1c6f1a)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:36.556509 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b296f2f0-5204-4ec9-b569-3da050dfffa1 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:36.558969 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-8f14c8e8-478e-4af2-b47d-535a29891c80 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:36.560174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-8f14c8e8-478e-4af2-b47d-535a29891c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:36.568316 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 938/1876] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:36.572105 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f14c8e8-478e-4af2-b47d-535a29891c80 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:36.583136 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f14c8e8-478e-4af2-b47d-535a29891c80 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:36.588990 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a77502-9d97-460d-8631-9daeb8e4eeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:36.589881 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a77502-9d97-460d-8631-9daeb8e4eeb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:36.590248 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a77502-9d97-460d-8631-9daeb8e4eeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:36.590631 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a77502-9d97-460d-8631-9daeb8e4eeb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:36.591805 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a77502-9d97-460d-8631-9daeb8e4eeb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:38:36.592165 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a77502-9d97-460d-8631-9daeb8e4eeb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:36.594175 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a77502-9d97-460d-8631-9daeb8e4eeb6 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:36.607170 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 939/1877] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:38:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:38:36.607300 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-b296f2f0-5204-4ec9-b569-3da050dfffa1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b7a6ac5b2eb445990db3811dd1c6f1a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:36.610838 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-b296f2f0-5204-4ec9-b569-3da050dfffa1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b7a6ac5b2eb445990db3811dd1c6f1a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:36.611344 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a77502-9d97-460d-8631-9daeb8e4eeb6 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:36.611496 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-b296f2f0-5204-4ec9-b569-3da050dfffa1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b7a6ac5b2eb445990db3811dd1c6f1a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:36.613012 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-01a77502-9d97-460d-8631-9daeb8e4eeb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:36.627264 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-b296f2f0-5204-4ec9-b569-3da050dfffa1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b7a6ac5b2eb445990db3811dd1c6f1a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:36.627426 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a77502-9d97-460d-8631-9daeb8e4eeb6 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:36.634471 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f98f1f-4b44-49d4-b7ab-ca10403cea56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:36.634650 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-b296f2f0-5204-4ec9-b569-3da050dfffa1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b7a6ac5b2eb445990db3811dd1c6f1a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:36.635274 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f98f1f-4b44-49d4-b7ab-ca10403cea56 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:36.635564 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f98f1f-4b44-49d4-b7ab-ca10403cea56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:36.635843 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f98f1f-4b44-49d4-b7ab-ca10403cea56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:36.636903 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f98f1f-4b44-49d4-b7ab-ca10403cea56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:36.640304 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f98f1f-4b44-49d4-b7ab-ca10403cea56 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:36.651475 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 939/1878] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:38:36 2022] DELETE /identity/v3/users/8b7a6ac5b2eb445990db3811dd1c6f1a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:36.655911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1cb6c0-e181-4313-b43a-ad1460913720 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:36.656212 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1cb6c0-e181-4313-b43a-ad1460913720 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:36.656493 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1cb6c0-e181-4313-b43a-ad1460913720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:36.680157 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 940/1879] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:36 2022] POST /identity/v3/projects => generated 367 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:36.680421 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-df1cb6c0-e181-4313-b43a-ad1460913720 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:36.681996 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-df1cb6c0-e181-4313-b43a-ad1460913720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:36.685551 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f98f1f-4b44-49d4-b7ab-ca10403cea56 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:36.692181 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:36.700499 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:36.760397 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f98f1f-4b44-49d4-b7ab-ca10403cea56 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:36.764267 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 940/1880] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:36.766416 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce72d1-f453-428c-9da1-c58cfe3986b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:36.767105 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce72d1-f453-428c-9da1-c58cfe3986b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:36.767391 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce72d1-f453-428c-9da1-c58cfe3986b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:36.767666 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce72d1-f453-428c-9da1-c58cfe3986b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:36.768390 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ce72d1-f453-428c-9da1-c58cfe3986b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:36.770327 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ce72d1-f453-428c-9da1-c58cfe3986b6 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:36.775113 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1cb6c0-e181-4313-b43a-ad1460913720 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:36.778191 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-df1cb6c0-e181-4313-b43a-ad1460913720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:36.797959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 941/1881] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:36 2022] POST /identity/v3/users => generated 284 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:36.801977 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce72d1-f453-428c-9da1-c58cfe3986b6 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:36.813825 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce72d1-f453-428c-9da1-c58cfe3986b6 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:36.818686 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad2b27-76ab-4ac4-909a-a00ead0d6ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:36.819360 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad2b27-76ab-4ac4-909a-a00ead0d6ec9 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:36.821641 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad2b27-76ab-4ac4-909a-a00ead0d6ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:36.821941 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad2b27-76ab-4ac4-909a-a00ead0d6ec9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:36.822864 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ad2b27-76ab-4ac4-909a-a00ead0d6ec9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:36.824571 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ad2b27-76ab-4ac4-909a-a00ead0d6ec9 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:36.839067 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 942/1882] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:36.840960 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad2b27-76ab-4ac4-909a-a00ead0d6ec9 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:36.851001 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad2b27-76ab-4ac4-909a-a00ead0d6ec9 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:36.863519 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0da853-f505-4f86-b233-5c7367683101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:36.864387 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0da853-f505-4f86-b233-5c7367683101 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:36.864730 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0da853-f505-4f86-b233-5c7367683101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:36.865071 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0da853-f505-4f86-b233-5c7367683101 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:36.865919 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0da853-f505-4f86-b233-5c7367683101 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:36.867550 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1cb6c0-e181-4313-b43a-ad1460913720 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:36.867660 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0da853-f505-4f86-b233-5c7367683101 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:36.871352 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915622ea-9ad6-401e-9cf1-73d41b079cb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:36.872021 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915622ea-9ad6-401e-9cf1-73d41b079cb4 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:36.872278 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915622ea-9ad6-401e-9cf1-73d41b079cb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:36.872553 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915622ea-9ad6-401e-9cf1-73d41b079cb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:36.882260 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 943/1883] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:36 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:38:36.884163 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0da853-f505-4f86-b233-5c7367683101 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:36.885794 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915622ea-9ad6-401e-9cf1-73d41b079cb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:36.887518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915622ea-9ad6-401e-9cf1-73d41b079cb4 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:36.896026 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0da853-f505-4f86-b233-5c7367683101 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:36.897487 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 941/1884] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:38:36.901910 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa85e30-4f59-4527-8cf1-48cbead8a26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:36.903562 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa85e30-4f59-4527-8cf1-48cbead8a26a admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:36.903839 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa85e30-4f59-4527-8cf1-48cbead8a26a admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:36.904472 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa85e30-4f59-4527-8cf1-48cbead8a26a admin admin] PATH_INFO: `/v3/projects/9a65513e7da94d70bfc8a749f0180da3/users/9172ea7d17bc4c17bf1b4bcfa66a8908/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:36.922444 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa85e30-4f59-4527-8cf1-48cbead8a26a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a65513e7da94d70bfc8a749f0180da3, user_id=9172ea7d17bc4c17bf1b4bcfa66a8908, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:36.925758 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa85e30-4f59-4527-8cf1-48cbead8a26a admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:36.949277 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 944/1885] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:36 2022] PUT /identity/v3/projects/9a65513e7da94d70bfc8a749f0180da3/users/9172ea7d17bc4c17bf1b4bcfa66a8908/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:36.961356 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744ba70-034b-4ac2-a237-bb6302a66842 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:36.961620 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744ba70-034b-4ac2-a237-bb6302a66842 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:36.961818 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744ba70-034b-4ac2-a237-bb6302a66842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:37.004555 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-9744ba70-034b-4ac2-a237-bb6302a66842 None None] MFA Rules not processed for user `9172ea7d17bc4c17bf1b4bcfa66a8908`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:37.006573 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-9744ba70-034b-4ac2-a237-bb6302a66842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:37.106844 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 942/1886] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:36 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:37.111343 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa85e30-4f59-4527-8cf1-48cbead8a26a admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:37.280705 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9744ba70-034b-4ac2-a237-bb6302a66842 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:37.282132 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa85e30-4f59-4527-8cf1-48cbead8a26a admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:37.282491 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-9744ba70-034b-4ac2-a237-bb6302a66842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:37.288358 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049c5641-2c36-4e62-8e32-7abe3de5cead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:37.289033 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c5641-2c36-4e62-8e32-7abe3de5cead admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:37.289318 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c5641-2c36-4e62-8e32-7abe3de5cead admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:37.289602 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c5641-2c36-4e62-8e32-7abe3de5cead admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:37.291079 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049c5641-2c36-4e62-8e32-7abe3de5cead admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:37.293549 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049c5641-2c36-4e62-8e32-7abe3de5cead admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:37.316707 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9744ba70-034b-4ac2-a237-bb6302a66842 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:37.323310 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a15e514-e4dd-4f8b-ba6a-47692d4299b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:37.325071 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a15e514-e4dd-4f8b-ba6a-47692d4299b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:37.325457 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a15e514-e4dd-4f8b-ba6a-47692d4299b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:37.325825 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a15e514-e4dd-4f8b-ba6a-47692d4299b8 admin admin] PATH_INFO: `/v3/projects/93a066a167a64858b708010591b14b1c` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:37.327251 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a15e514-e4dd-4f8b-ba6a-47692d4299b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93a066a167a64858b708010591b14b1c)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:37.329379 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a15e514-e4dd-4f8b-ba6a-47692d4299b8 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:37.354477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 945/1887] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:37 2022] POST /identity/v3/projects => generated 367 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:37.356228 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049c5641-2c36-4e62-8e32-7abe3de5cead admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:37.416010 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:37.425249 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:37.428006 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7a15e514-e4dd-4f8b-ba6a-47692d4299b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93a066a167a64858b708010591b14b1c', 'request_id': 'req-7a15e514-e4dd-4f8b-ba6a-47692d4299b8'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:37.443374 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7a15e514-e4dd-4f8b-ba6a-47692d4299b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93a066a167a64858b708010591b14b1c', 'request_id': 'req-7a15e514-e4dd-4f8b-ba6a-47692d4299b8'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:37.453431 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7a15e514-e4dd-4f8b-ba6a-47692d4299b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93a066a167a64858b708010591b14b1c', 'request_id': 'req-7a15e514-e4dd-4f8b-ba6a-47692d4299b8'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:37.462399 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049c5641-2c36-4e62-8e32-7abe3de5cead admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:37.467868 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7a15e514-e4dd-4f8b-ba6a-47692d4299b8 admin admin] The token cache is being invalidate because project 93a066a167a64858b708010591b14b1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:37.468240 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7a15e514-e4dd-4f8b-ba6a-47692d4299b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:37.469040 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7a15e514-e4dd-4f8b-ba6a-47692d4299b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:37.478244 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 943/1888] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:37 2022] DELETE /identity/v3/projects/93a066a167a64858b708010591b14b1c => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:38:37.479381 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a15e514-e4dd-4f8b-ba6a-47692d4299b8 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:37.481609 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7a15e514-e4dd-4f8b-ba6a-47692d4299b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:37.588849 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643a447-a00c-4060-85fc-35046a73c7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:37.589692 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643a447-a00c-4060-85fc-35046a73c7eb admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:37.590528 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643a447-a00c-4060-85fc-35046a73c7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:37.591059 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643a447-a00c-4060-85fc-35046a73c7eb admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:37.592543 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4643a447-a00c-4060-85fc-35046a73c7eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:37.594902 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4643a447-a00c-4060-85fc-35046a73c7eb admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:37.605638 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a15e514-e4dd-4f8b-ba6a-47692d4299b8 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:37.611354 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f067b-90e6-4cda-9471-6d68a690096b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:37.612374 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f067b-90e6-4cda-9471-6d68a690096b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:37.613182 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f067b-90e6-4cda-9471-6d68a690096b admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:37.613182 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f067b-90e6-4cda-9471-6d68a690096b admin admin] PATH_INFO: `/v3/users/7442a252a8cc4c25a1c889b60167dce2` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:37.614383 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f067b-90e6-4cda-9471-6d68a690096b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7442a252a8cc4c25a1c889b60167dce2)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:37.616670 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f067b-90e6-4cda-9471-6d68a690096b admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:37.633126 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 946/1889] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:37 2022] POST /identity/v3/users => generated 290 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:37.637513 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643a447-a00c-4060-85fc-35046a73c7eb admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:37.643830 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-4643a447-a00c-4060-85fc-35046a73c7eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:37.666318 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643a447-a00c-4060-85fc-35046a73c7eb admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:37.671221 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-194f067b-90e6-4cda-9471-6d68a690096b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7442a252a8cc4c25a1c889b60167dce2'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:37.672074 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-194f067b-90e6-4cda-9471-6d68a690096b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7442a252a8cc4c25a1c889b60167dce2'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:37.672979 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-194f067b-90e6-4cda-9471-6d68a690096b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7442a252a8cc4c25a1c889b60167dce2'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:37.674545 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b9815-afaa-4b88-979b-d558e00707f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:37.676835 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b9815-afaa-4b88-979b-d558e00707f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:37.677189 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b9815-afaa-4b88-979b-d558e00707f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:37.677887 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b9815-afaa-4b88-979b-d558e00707f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:37.679054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5b9815-afaa-4b88-979b-d558e00707f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:37.681916 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5b9815-afaa-4b88-979b-d558e00707f6 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:37.687080 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-194f067b-90e6-4cda-9471-6d68a690096b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7442a252a8cc4c25a1c889b60167dce2'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:37.697245 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 947/1890] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:37 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:37.698489 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b9815-afaa-4b88-979b-d558e00707f6 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:37.700036 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-194f067b-90e6-4cda-9471-6d68a690096b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7442a252a8cc4c25a1c889b60167dce2'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:37.701138 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-4b5b9815-afaa-4b88-979b-d558e00707f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:37.715794 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b9815-afaa-4b88-979b-d558e00707f6 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:37.718260 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 944/1891] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:38:37 2022] DELETE /identity/v3/users/7442a252a8cc4c25a1c889b60167dce2 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:37.785236 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd345e9-09b5-4aa3-857f-79bc21e67129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:37.786333 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd345e9-09b5-4aa3-857f-79bc21e67129 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:37.786675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd345e9-09b5-4aa3-857f-79bc21e67129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:37.787028 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd345e9-09b5-4aa3-857f-79bc21e67129 admin admin] PATH_INFO: `/v3/projects/01199c800bed442bbb84de9d8d66eb50/users/63f35cc62c7a4bffb4c155e144c225f8/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:37.807138 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd345e9-09b5-4aa3-857f-79bc21e67129 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01199c800bed442bbb84de9d8d66eb50, user_id=63f35cc62c7a4bffb4c155e144c225f8, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:37.810886 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd345e9-09b5-4aa3-857f-79bc21e67129 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:37.829152 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 948/1892] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:37 2022] PUT /identity/v3/projects/01199c800bed442bbb84de9d8d66eb50/users/63f35cc62c7a4bffb4c155e144c225f8/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:37.833247 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f067b-90e6-4cda-9471-6d68a690096b admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:37.860609 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f067b-90e6-4cda-9471-6d68a690096b admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:37.965804 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:37.981902 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b09be-ea13-4a44-ac81-7f6ee386f268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:37.982641 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b09be-ea13-4a44-ac81-7f6ee386f268 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:37.982937 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b09be-ea13-4a44-ac81-7f6ee386f268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:37.983232 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b09be-ea13-4a44-ac81-7f6ee386f268 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:37.983936 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140b09be-ea13-4a44-ac81-7f6ee386f268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:37.985624 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140b09be-ea13-4a44-ac81-7f6ee386f268 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:37.998439 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 945/1893] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:37 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:38.002860 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd345e9-09b5-4aa3-857f-79bc21e67129 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:38.015071 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd345e9-09b5-4aa3-857f-79bc21e67129 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:38.020184 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e7df9-de10-4e90-886c-3a79c6033f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:38.021161 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e7df9-de10-4e90-886c-3a79c6033f1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:38.021578 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e7df9-de10-4e90-886c-3a79c6033f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:38.022676 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e7df9-de10-4e90-886c-3a79c6033f1e admin admin] PATH_INFO: `/v3/projects/01199c800bed442bbb84de9d8d66eb50/users/63f35cc62c7a4bffb4c155e144c225f8/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:38.024877 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e7df9-de10-4e90-886c-3a79c6033f1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=01199c800bed442bbb84de9d8d66eb50, user_id=63f35cc62c7a4bffb4c155e144c225f8, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:38.028473 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e7df9-de10-4e90-886c-3a79c6033f1e admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:38.042119 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 949/1894] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:38 2022] PUT /identity/v3/projects/01199c800bed442bbb84de9d8d66eb50/users/63f35cc62c7a4bffb4c155e144c225f8/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:38:38.052072 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215fc02c-7e2b-4d2c-95bc-abc099ca63f5 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:38.052440 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215fc02c-7e2b-4d2c-95bc-abc099ca63f5 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:38.052634 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215fc02c-7e2b-4d2c-95bc-abc099ca63f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:38.082941 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-215fc02c-7e2b-4d2c-95bc-abc099ca63f5 None None] MFA Rules not processed for user `63f35cc62c7a4bffb4c155e144c225f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:38.084165 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-215fc02c-7e2b-4d2c-95bc-abc099ca63f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:38.142245 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e7df9-de10-4e90-886c-3a79c6033f1e admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:38.144013 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-be5e7df9-de10-4e90-886c-3a79c6033f1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:38.173747 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 946/1895] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:38 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:38.188943 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215fc02c-7e2b-4d2c-95bc-abc099ca63f5 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:38.191172 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-215fc02c-7e2b-4d2c-95bc-abc099ca63f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:38.217511 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:38.256232 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e7df9-de10-4e90-886c-3a79c6033f1e admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:38.262813 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419bbd1-c66f-4b4d-a909-7ff2537b0632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:38.263799 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419bbd1-c66f-4b4d-a909-7ff2537b0632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:38.264174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419bbd1-c66f-4b4d-a909-7ff2537b0632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:38.264896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419bbd1-c66f-4b4d-a909-7ff2537b0632 admin admin] PATH_INFO: `/v3/projects/3b40132403f44a719658d58e30f7b464` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:38.266895 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c419bbd1-c66f-4b4d-a909-7ff2537b0632 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b40132403f44a719658d58e30f7b464)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:38.269909 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c419bbd1-c66f-4b4d-a909-7ff2537b0632 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:38.282256 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215fc02c-7e2b-4d2c-95bc-abc099ca63f5 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:38.286738 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb132e-1743-4473-9f64-191d2b0557d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:38.287396 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb132e-1743-4473-9f64-191d2b0557d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:38.287685 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb132e-1743-4473-9f64-191d2b0557d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:38.287968 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb132e-1743-4473-9f64-191d2b0557d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:38.299007 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bb132e-1743-4473-9f64-191d2b0557d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:38.300669 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bb132e-1743-4473-9f64-191d2b0557d4 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:38.312410 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 947/1896] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:38 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:38:38.345569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-c419bbd1-c66f-4b4d-a909-7ff2537b0632 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b40132403f44a719658d58e30f7b464', 'request_id': 'req-c419bbd1-c66f-4b4d-a909-7ff2537b0632'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:38.357887 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-c419bbd1-c66f-4b4d-a909-7ff2537b0632 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b40132403f44a719658d58e30f7b464', 'request_id': 'req-c419bbd1-c66f-4b4d-a909-7ff2537b0632'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:38.369012 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-c419bbd1-c66f-4b4d-a909-7ff2537b0632 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b40132403f44a719658d58e30f7b464', 'request_id': 'req-c419bbd1-c66f-4b4d-a909-7ff2537b0632'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:38.400146 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-c419bbd1-c66f-4b4d-a909-7ff2537b0632 admin admin] The token cache is being invalidate because project 3b40132403f44a719658d58e30f7b464 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:38.400543 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-c419bbd1-c66f-4b4d-a909-7ff2537b0632 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:38.401333 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-c419bbd1-c66f-4b4d-a909-7ff2537b0632 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:38.405736 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 950/1897] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:38 2022] DELETE /identity/v3/projects/3b40132403f44a719658d58e30f7b464 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:38:38.407206 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb132e-1743-4473-9f64-191d2b0557d4 service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:38.436786 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb132e-1743-4473-9f64-191d2b0557d4 service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:38.545342 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d22540-cf7e-4ec1-8dd2-aa9e82a1cbe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:38.546396 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d22540-cf7e-4ec1-8dd2-aa9e82a1cbe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:38.546787 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d22540-cf7e-4ec1-8dd2-aa9e82a1cbe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:38.547076 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d22540-cf7e-4ec1-8dd2-aa9e82a1cbe2 admin admin] PATH_INFO: `/v3/users/4545dad1b2ad497c9470c36986901d1e` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:38.548869 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d22540-cf7e-4ec1-8dd2-aa9e82a1cbe2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4545dad1b2ad497c9470c36986901d1e)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:38.550927 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d22540-cf7e-4ec1-8dd2-aa9e82a1cbe2 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:38.574467 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:38.622923 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-04d22540-cf7e-4ec1-8dd2-aa9e82a1cbe2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4545dad1b2ad497c9470c36986901d1e'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:38.624073 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-04d22540-cf7e-4ec1-8dd2-aa9e82a1cbe2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4545dad1b2ad497c9470c36986901d1e'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:38.625124 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-04d22540-cf7e-4ec1-8dd2-aa9e82a1cbe2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4545dad1b2ad497c9470c36986901d1e'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:38.647486 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-04d22540-cf7e-4ec1-8dd2-aa9e82a1cbe2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4545dad1b2ad497c9470c36986901d1e'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:38.662158 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-04d22540-cf7e-4ec1-8dd2-aa9e82a1cbe2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4545dad1b2ad497c9470c36986901d1e'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:38.702951 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 948/1898] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:38:38 2022] DELETE /identity/v3/users/4545dad1b2ad497c9470c36986901d1e => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:39.376210 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419bbd1-c66f-4b4d-a909-7ff2537b0632 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:39.377654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-c419bbd1-c66f-4b4d-a909-7ff2537b0632 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:39.393579 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419bbd1-c66f-4b4d-a909-7ff2537b0632 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:39.524559 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9015ae11-10f6-4ac7-98c7-a0d9e5586f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:39.525699 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015ae11-10f6-4ac7-98c7-a0d9e5586f5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:39.525699 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015ae11-10f6-4ac7-98c7-a0d9e5586f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:39.525998 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015ae11-10f6-4ac7-98c7-a0d9e5586f5d admin admin] PATH_INFO: `/v3/projects/b504351549004bf29fb94093a6e814a2` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:39.527520 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9015ae11-10f6-4ac7-98c7-a0d9e5586f5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b504351549004bf29fb94093a6e814a2)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:39.529972 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9015ae11-10f6-4ac7-98c7-a0d9e5586f5d admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:39.604843 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-9015ae11-10f6-4ac7-98c7-a0d9e5586f5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b504351549004bf29fb94093a6e814a2', 'request_id': 'req-9015ae11-10f6-4ac7-98c7-a0d9e5586f5d'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:39.619510 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-9015ae11-10f6-4ac7-98c7-a0d9e5586f5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b504351549004bf29fb94093a6e814a2', 'request_id': 'req-9015ae11-10f6-4ac7-98c7-a0d9e5586f5d'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:39.634528 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-9015ae11-10f6-4ac7-98c7-a0d9e5586f5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b504351549004bf29fb94093a6e814a2', 'request_id': 'req-9015ae11-10f6-4ac7-98c7-a0d9e5586f5d'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:39.637470 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:39.660650 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-9015ae11-10f6-4ac7-98c7-a0d9e5586f5d admin admin] The token cache is being invalidate because project b504351549004bf29fb94093a6e814a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:39.661436 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-9015ae11-10f6-4ac7-98c7-a0d9e5586f5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:39.663043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-9015ae11-10f6-4ac7-98c7-a0d9e5586f5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:39.666366 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 951/1899] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:39 2022] DELETE /identity/v3/projects/b504351549004bf29fb94093a6e814a2 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:39.672498 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23a468-7681-403a-9951-ce0a1e8a9739 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:39.673630 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23a468-7681-403a-9951-ce0a1e8a9739 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:39.674194 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23a468-7681-403a-9951-ce0a1e8a9739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:39.704200 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-8c23a468-7681-403a-9951-ce0a1e8a9739 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:39.706070 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8c23a468-7681-403a-9951-ce0a1e8a9739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:39.810644 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 949/1900] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:39.812765 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19949545-9f69-4546-a0c5-88fbb57cf52d None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:39.813138 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19949545-9f69-4546-a0c5-88fbb57cf52d None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:39.813556 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19949545-9f69-4546-a0c5-88fbb57cf52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:39.866658 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-19949545-9f69-4546-a0c5-88fbb57cf52d None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:39.868592 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-19949545-9f69-4546-a0c5-88fbb57cf52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:39.888826 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 952/1901] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:39.893296 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c23a468-7681-403a-9951-ce0a1e8a9739 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:39.910361 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c23a468-7681-403a-9951-ce0a1e8a9739 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:39.917078 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b537795-9236-475a-8c31-05861cb36fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:39.917934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b537795-9236-475a-8c31-05861cb36fd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:39.918263 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b537795-9236-475a-8c31-05861cb36fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:39.918544 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b537795-9236-475a-8c31-05861cb36fd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:39.919378 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b537795-9236-475a-8c31-05861cb36fd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:38:39.919698 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b537795-9236-475a-8c31-05861cb36fd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:39.921311 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b537795-9236-475a-8c31-05861cb36fd3 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:39.955395 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 950/1902] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:38:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:38:39.956633 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19949545-9f69-4546-a0c5-88fbb57cf52d None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:39.972576 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19949545-9f69-4546-a0c5-88fbb57cf52d None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:39.986111 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b427e4-5f2e-42fb-80ac-3f5d9eac9951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:39.986816 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b427e4-5f2e-42fb-80ac-3f5d9eac9951 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:39.987101 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b427e4-5f2e-42fb-80ac-3f5d9eac9951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:39.987381 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b427e4-5f2e-42fb-80ac-3f5d9eac9951 admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:39.988097 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b427e4-5f2e-42fb-80ac-3f5d9eac9951 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:39.990497 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b427e4-5f2e-42fb-80ac-3f5d9eac9951 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:40.052909 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 953/1903] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:39 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:40.056995 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b537795-9236-475a-8c31-05861cb36fd3 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:40.074890 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b537795-9236-475a-8c31-05861cb36fd3 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:40.198322 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8ee70-8e11-4c6a-95a6-fda8f5bfc330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:40.199976 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8ee70-8e11-4c6a-95a6-fda8f5bfc330 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:40.200362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8ee70-8e11-4c6a-95a6-fda8f5bfc330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:40.200650 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8ee70-8e11-4c6a-95a6-fda8f5bfc330 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:40.202013 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f8ee70-8e11-4c6a-95a6-fda8f5bfc330 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:40.204473 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f8ee70-8e11-4c6a-95a6-fda8f5bfc330 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:40.237786 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 951/1904] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:40 2022] POST /identity/v3/users => generated 292 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:40.241855 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b427e4-5f2e-42fb-80ac-3f5d9eac9951 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:40.258533 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b427e4-5f2e-42fb-80ac-3f5d9eac9951 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:40.267343 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65beea8d-0f79-4845-96db-359d7d84da84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:40.268437 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65beea8d-0f79-4845-96db-359d7d84da84 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:40.268867 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65beea8d-0f79-4845-96db-359d7d84da84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:40.270399 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65beea8d-0f79-4845-96db-359d7d84da84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:40.272173 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65beea8d-0f79-4845-96db-359d7d84da84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:40.278366 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65beea8d-0f79-4845-96db-359d7d84da84 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:40.306195 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 954/1905] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:40.310121 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8ee70-8e11-4c6a-95a6-fda8f5bfc330 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:40.320685 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8ee70-8e11-4c6a-95a6-fda8f5bfc330 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:40.327918 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3e9989-97b4-4993-b2fc-381e7ae23b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:40.328857 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e9989-97b4-4993-b2fc-381e7ae23b9b admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:40.329375 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e9989-97b4-4993-b2fc-381e7ae23b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:40.329805 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e9989-97b4-4993-b2fc-381e7ae23b9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:40.330775 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3e9989-97b4-4993-b2fc-381e7ae23b9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:40.332601 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3e9989-97b4-4993-b2fc-381e7ae23b9b admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:40.335327 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:40.346746 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 952/1906] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:40.350365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65beea8d-0f79-4845-96db-359d7d84da84 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:40.363470 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65beea8d-0f79-4845-96db-359d7d84da84 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:40.368904 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a36217-80ab-4ea9-a991-54e3ed797f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:40.369800 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a36217-80ab-4ea9-a991-54e3ed797f50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:40.370104 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a36217-80ab-4ea9-a991-54e3ed797f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:40.370431 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a36217-80ab-4ea9-a991-54e3ed797f50 admin admin] PATH_INFO: `/v3/projects/574db4fd8c7c4742918c3628a3cfa993/users/2589331db7ba4ed3904df67e89b9f896/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:40.380246 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3e9989-97b4-4993-b2fc-381e7ae23b9b admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:40.382149 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ae3e9989-97b4-4993-b2fc-381e7ae23b9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:40.388174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a36217-80ab-4ea9-a991-54e3ed797f50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=574db4fd8c7c4742918c3628a3cfa993, user_id=2589331db7ba4ed3904df67e89b9f896, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:40.391155 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a36217-80ab-4ea9-a991-54e3ed797f50 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:40.397490 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3e9989-97b4-4993-b2fc-381e7ae23b9b admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:40.403544 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c1c2c7-a922-4d13-81cd-2dc7a116eee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:40.404669 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1c2c7-a922-4d13-81cd-2dc7a116eee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:40.404961 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1c2c7-a922-4d13-81cd-2dc7a116eee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:40.405232 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1c2c7-a922-4d13-81cd-2dc7a116eee2 admin admin] PATH_INFO: `/v3/users/9172ea7d17bc4c17bf1b4bcfa66a8908` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:40.406523 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c1c2c7-a922-4d13-81cd-2dc7a116eee2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9172ea7d17bc4c17bf1b4bcfa66a8908)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:40.416458 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 955/1907] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:40 2022] PUT /identity/v3/projects/574db4fd8c7c4742918c3628a3cfa993/users/2589331db7ba4ed3904df67e89b9f896/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:38:40.416645 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c1c2c7-a922-4d13-81cd-2dc7a116eee2 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:40.420083 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7c0c0-9bb8-46b2-a10f-e70d4c430f21 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:40.420668 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7c0c0-9bb8-46b2-a10f-e70d4c430f21 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:40.421236 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7c0c0-9bb8-46b2-a10f-e70d4c430f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:40.453765 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-84d7c0c0-9bb8-46b2-a10f-e70d4c430f21 None None] MFA Rules not processed for user `2589331db7ba4ed3904df67e89b9f896`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:40.455247 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-84d7c0c0-9bb8-46b2-a10f-e70d4c430f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:40.555540 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 956/1908] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:38:40.558164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d7c0c0-9bb8-46b2-a10f-e70d4c430f21 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:40.613627 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-33c1c2c7-a922-4d13-81cd-2dc7a116eee2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9172ea7d17bc4c17bf1b4bcfa66a8908'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:40.615027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-33c1c2c7-a922-4d13-81cd-2dc7a116eee2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9172ea7d17bc4c17bf1b4bcfa66a8908'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:40.615572 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-33c1c2c7-a922-4d13-81cd-2dc7a116eee2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9172ea7d17bc4c17bf1b4bcfa66a8908'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:40.637056 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-33c1c2c7-a922-4d13-81cd-2dc7a116eee2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9172ea7d17bc4c17bf1b4bcfa66a8908'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:40.646495 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-33c1c2c7-a922-4d13-81cd-2dc7a116eee2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9172ea7d17bc4c17bf1b4bcfa66a8908'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:40.650527 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d7c0c0-9bb8-46b2-a10f-e70d4c430f21 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:40.657244 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e6131-79ae-4ff8-b5e8-6e20b25085f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:40.658025 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e6131-79ae-4ff8-b5e8-6e20b25085f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:40.658277 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e6131-79ae-4ff8-b5e8-6e20b25085f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:40.658571 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e6131-79ae-4ff8-b5e8-6e20b25085f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:40.659338 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e6131-79ae-4ff8-b5e8-6e20b25085f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:40.661523 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e6131-79ae-4ff8-b5e8-6e20b25085f6 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:40.669861 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 953/1909] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:38:40 2022] DELETE /identity/v3/users/9172ea7d17bc4c17bf1b4bcfa66a8908 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:38:40.675657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86aa08-5975-4626-97f9-001fa0ff7ee4 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:40.676250 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86aa08-5975-4626-97f9-001fa0ff7ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:40.676250 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86aa08-5975-4626-97f9-001fa0ff7ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:40.693880 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 957/1910] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:40 2022] POST /identity/v3/projects => generated 383 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:38:40.699954 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e6131-79ae-4ff8-b5e8-6e20b25085f6 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:40.704474 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-f54e6131-79ae-4ff8-b5e8-6e20b25085f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:40.705057 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-8b86aa08-5975-4626-97f9-001fa0ff7ee4 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:40.706413 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8b86aa08-5975-4626-97f9-001fa0ff7ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:40.709667 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:40.773476 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 954/1911] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:40.775719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e6131-79ae-4ff8-b5e8-6e20b25085f6 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:40.781148 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0c4913-763d-4cd5-be10-82adee8548a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:40.781887 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c4913-763d-4cd5-be10-82adee8548a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:40.782273 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c4913-763d-4cd5-be10-82adee8548a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:40.782633 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c4913-763d-4cd5-be10-82adee8548a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:40.783399 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0c4913-763d-4cd5-be10-82adee8548a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:40.785293 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86aa08-5975-4626-97f9-001fa0ff7ee4 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:40.785839 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0c4913-763d-4cd5-be10-82adee8548a0 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:40.786888 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8b86aa08-5975-4626-97f9-001fa0ff7ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:40.809926 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 958/1912] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:40 2022] POST /identity/v3/users => generated 298 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:38:40.813443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0c4913-763d-4cd5-be10-82adee8548a0 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:40.824017 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0c4913-763d-4cd5-be10-82adee8548a0 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:40.830132 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cb945d-199d-4d44-be23-00e6299f1d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:40.830889 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb945d-199d-4d44-be23-00e6299f1d28 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:40.831177 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb945d-199d-4d44-be23-00e6299f1d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:40.831501 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb945d-199d-4d44-be23-00e6299f1d28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:40.832239 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cb945d-199d-4d44-be23-00e6299f1d28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:40.834072 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cb945d-199d-4d44-be23-00e6299f1d28 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:40.845706 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 959/1913] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:38:40.849582 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cb945d-199d-4d44-be23-00e6299f1d28 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:40.853888 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86aa08-5975-4626-97f9-001fa0ff7ee4 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:40.860690 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78494c49-d241-4667-b1dd-47af5da9330c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:40.861350 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78494c49-d241-4667-b1dd-47af5da9330c service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:40.861632 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78494c49-d241-4667-b1dd-47af5da9330c service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:40.862107 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78494c49-d241-4667-b1dd-47af5da9330c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:40.871972 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cb945d-199d-4d44-be23-00e6299f1d28 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:40.874980 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78494c49-d241-4667-b1dd-47af5da9330c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:40.876758 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78494c49-d241-4667-b1dd-47af5da9330c service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:40.885725 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58e1ac-f3ab-4d69-af36-851066d437fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:40.887795 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58e1ac-f3ab-4d69-af36-851066d437fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:40.889511 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58e1ac-f3ab-4d69-af36-851066d437fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:40.889717 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58e1ac-f3ab-4d69-af36-851066d437fd admin admin] PATH_INFO: `/v3/projects/a77989ad54e24c4097fbfc6b6af445b9/users/416fb78b591041e09ef42b08534bbf04/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:40.903903 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 955/1914] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:38:40.917364 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c58e1ac-f3ab-4d69-af36-851066d437fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a77989ad54e24c4097fbfc6b6af445b9, user_id=416fb78b591041e09ef42b08534bbf04, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:40.921579 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c58e1ac-f3ab-4d69-af36-851066d437fd admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:40.945813 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 960/1915] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:40 2022] PUT /identity/v3/projects/a77989ad54e24c4097fbfc6b6af445b9/users/416fb78b591041e09ef42b08534bbf04/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:38:40.949787 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78494c49-d241-4667-b1dd-47af5da9330c service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:40.962219 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78494c49-d241-4667-b1dd-47af5da9330c service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:40.988016 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:41.065374 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9568016a-4e4a-4e75-b43b-ef21131fcc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:41.066222 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568016a-4e4a-4e75-b43b-ef21131fcc87 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:41.066515 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568016a-4e4a-4e75-b43b-ef21131fcc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:41.067131 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568016a-4e4a-4e75-b43b-ef21131fcc87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:41.067935 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9568016a-4e4a-4e75-b43b-ef21131fcc87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:41.069634 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9568016a-4e4a-4e75-b43b-ef21131fcc87 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:41.091285 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 956/1916] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:41.092504 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58e1ac-f3ab-4d69-af36-851066d437fd admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:41.104251 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58e1ac-f3ab-4d69-af36-851066d437fd admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:41.109686 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad02325-e042-4880-bbc5-cd79d0d87e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:41.110717 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad02325-e042-4880-bbc5-cd79d0d87e4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:41.111103 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad02325-e042-4880-bbc5-cd79d0d87e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:41.111448 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad02325-e042-4880-bbc5-cd79d0d87e4f admin admin] PATH_INFO: `/v3/projects/a77989ad54e24c4097fbfc6b6af445b9/users/416fb78b591041e09ef42b08534bbf04/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:41.114132 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad02325-e042-4880-bbc5-cd79d0d87e4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a77989ad54e24c4097fbfc6b6af445b9, user_id=416fb78b591041e09ef42b08534bbf04, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:41.116921 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad02325-e042-4880-bbc5-cd79d0d87e4f admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:41.129751 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 961/1917] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:41 2022] PUT /identity/v3/projects/a77989ad54e24c4097fbfc6b6af445b9/users/416fb78b591041e09ef42b08534bbf04/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:41.141056 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242db5b-2b7a-4a36-97d2-1957578d447f None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:41.141056 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242db5b-2b7a-4a36-97d2-1957578d447f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:41.141303 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242db5b-2b7a-4a36-97d2-1957578d447f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:41.169924 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-7242db5b-2b7a-4a36-97d2-1957578d447f None None] MFA Rules not processed for user `416fb78b591041e09ef42b08534bbf04`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:41.171407 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7242db5b-2b7a-4a36-97d2-1957578d447f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:41.256277 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 957/1918] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:41.267389 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad02325-e042-4880-bbc5-cd79d0d87e4f admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:41.269201 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-7ad02325-e042-4880-bbc5-cd79d0d87e4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:41.285410 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242db5b-2b7a-4a36-97d2-1957578d447f None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:41.286933 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7242db5b-2b7a-4a36-97d2-1957578d447f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:41.399847 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad02325-e042-4880-bbc5-cd79d0d87e4f admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:41.405713 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05eeae-9784-412d-a724-a4989c577ebf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:41.406564 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05eeae-9784-412d-a724-a4989c577ebf service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:41.406928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05eeae-9784-412d-a724-a4989c577ebf service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:41.407247 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242db5b-2b7a-4a36-97d2-1957578d447f None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:41.407364 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05eeae-9784-412d-a724-a4989c577ebf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:41.411639 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-8b05eeae-9784-412d-a724-a4989c577ebf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:41.412685 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10faef0c-de6f-43cb-aa69-bef466ac86cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:41.413620 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10faef0c-de6f-43cb-aa69-bef466ac86cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:41.413973 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10faef0c-de6f-43cb-aa69-bef466ac86cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:41.414340 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10faef0c-de6f-43cb-aa69-bef466ac86cf admin admin] PATH_INFO: `/v3/projects/9a65513e7da94d70bfc8a749f0180da3` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:41.415623 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10faef0c-de6f-43cb-aa69-bef466ac86cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a65513e7da94d70bfc8a749f0180da3)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:41.418609 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10faef0c-de6f-43cb-aa69-bef466ac86cf admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:41.463055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:41.477741 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-10faef0c-de6f-43cb-aa69-bef466ac86cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a65513e7da94d70bfc8a749f0180da3', 'request_id': 'req-10faef0c-de6f-43cb-aa69-bef466ac86cf'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:41.499094 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-10faef0c-de6f-43cb-aa69-bef466ac86cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a65513e7da94d70bfc8a749f0180da3', 'request_id': 'req-10faef0c-de6f-43cb-aa69-bef466ac86cf'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:41.518307 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-10faef0c-de6f-43cb-aa69-bef466ac86cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a65513e7da94d70bfc8a749f0180da3', 'request_id': 'req-10faef0c-de6f-43cb-aa69-bef466ac86cf'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:41.555083 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-10faef0c-de6f-43cb-aa69-bef466ac86cf admin admin] The token cache is being invalidate because project 9a65513e7da94d70bfc8a749f0180da3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:41.558787 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-10faef0c-de6f-43cb-aa69-bef466ac86cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:41.560289 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-10faef0c-de6f-43cb-aa69-bef466ac86cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:41.562811 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 958/1919] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:41 2022] DELETE /identity/v3/projects/9a65513e7da94d70bfc8a749f0180da3 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:38:41.566967 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10faef0c-de6f-43cb-aa69-bef466ac86cf admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:41.569731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-10faef0c-de6f-43cb-aa69-bef466ac86cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:41.577344 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b05eeae-9784-412d-a724-a4989c577ebf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:41.579199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b05eeae-9784-412d-a724-a4989c577ebf service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:41.587955 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:41.602989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 962/1920] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:38:41.660384 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05eeae-9784-412d-a724-a4989c577ebf service cinder] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:41.662217 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-8b05eeae-9784-412d-a724-a4989c577ebf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:41.677728 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05eeae-9784-412d-a724-a4989c577ebf service cinder] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:41.717733 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10faef0c-de6f-43cb-aa69-bef466ac86cf admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:41.723664 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e8b442-56bc-4c02-896a-d0221780843c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:41.724856 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8b442-56bc-4c02-896a-d0221780843c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:41.725167 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8b442-56bc-4c02-896a-d0221780843c admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:41.725426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8b442-56bc-4c02-896a-d0221780843c admin admin] PATH_INFO: `/v3/users/63f35cc62c7a4bffb4c155e144c225f8` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:41.726708 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e8b442-56bc-4c02-896a-d0221780843c admin admin] RBAC: Authorizing `identity:delete_user(user_id=63f35cc62c7a4bffb4c155e144c225f8)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:41.728960 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e8b442-56bc-4c02-896a-d0221780843c admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:41.786343 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a63a9-a647-461a-83b2-0359d9be45a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:41.787233 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a63a9-a647-461a-83b2-0359d9be45a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:41.787764 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a63a9-a647-461a-83b2-0359d9be45a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:41.787853 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a63a9-a647-461a-83b2-0359d9be45a0 admin admin] PATH_INFO: `/v3/users/56e6903cec2640cb8da3d62c45116820` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:41.789305 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a63a9-a647-461a-83b2-0359d9be45a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56e6903cec2640cb8da3d62c45116820)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:41.791920 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a63a9-a647-461a-83b2-0359d9be45a0 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:41.796397 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-a8e8b442-56bc-4c02-896a-d0221780843c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63f35cc62c7a4bffb4c155e144c225f8'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:41.797709 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-a8e8b442-56bc-4c02-896a-d0221780843c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63f35cc62c7a4bffb4c155e144c225f8'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:41.798838 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-a8e8b442-56bc-4c02-896a-d0221780843c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63f35cc62c7a4bffb4c155e144c225f8'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:41.814707 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-a8e8b442-56bc-4c02-896a-d0221780843c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63f35cc62c7a4bffb4c155e144c225f8'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:41.829616 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-a8e8b442-56bc-4c02-896a-d0221780843c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63f35cc62c7a4bffb4c155e144c225f8'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:41.856558 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 959/1921] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:38:41 2022] DELETE /identity/v3/users/63f35cc62c7a4bffb4c155e144c225f8 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:41.871333 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-b01a63a9-a647-461a-83b2-0359d9be45a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56e6903cec2640cb8da3d62c45116820'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:41.872224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-b01a63a9-a647-461a-83b2-0359d9be45a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56e6903cec2640cb8da3d62c45116820'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:41.887236 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-b01a63a9-a647-461a-83b2-0359d9be45a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56e6903cec2640cb8da3d62c45116820'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:41.888357 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-b01a63a9-a647-461a-83b2-0359d9be45a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56e6903cec2640cb8da3d62c45116820'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:41.903305 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-b01a63a9-a647-461a-83b2-0359d9be45a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56e6903cec2640cb8da3d62c45116820'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:41.937011 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 963/1922] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:38:41 2022] DELETE /identity/v3/users/56e6903cec2640cb8da3d62c45116820 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:38:41.944443 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ed326-2d70-43aa-92d7-ea3346144646 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:41.944735 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ed326-2d70-43aa-92d7-ea3346144646 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:41.945088 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ed326-2d70-43aa-92d7-ea3346144646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:41.975140 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-189ed326-2d70-43aa-92d7-ea3346144646 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:41.978733 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-189ed326-2d70-43aa-92d7-ea3346144646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:42.039071 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:42.093590 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 960/1923] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:42.120550 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a63a9-a647-461a-83b2-0359d9be45a0 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:42.124250 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-b01a63a9-a647-461a-83b2-0359d9be45a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:42.355624 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ed326-2d70-43aa-92d7-ea3346144646 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:42.357693 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-189ed326-2d70-43aa-92d7-ea3346144646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:42.361994 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a63a9-a647-461a-83b2-0359d9be45a0 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:42.374908 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406f74ad-995c-42f7-b36e-31ef881a7f6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:42.375850 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f74ad-995c-42f7-b36e-31ef881a7f6e service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:42.376129 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f74ad-995c-42f7-b36e-31ef881a7f6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:42.376406 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f74ad-995c-42f7-b36e-31ef881a7f6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:42.380051 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ed326-2d70-43aa-92d7-ea3346144646 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:42.403377 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda6eb2f-e173-4c63-a617-3b51692e6db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:42.404660 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6eb2f-e173-4c63-a617-3b51692e6db5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:42.405107 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6eb2f-e173-4c63-a617-3b51692e6db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:42.405535 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6eb2f-e173-4c63-a617-3b51692e6db5 admin admin] PATH_INFO: `/v3/projects/01199c800bed442bbb84de9d8d66eb50` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:42.419167 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda6eb2f-e173-4c63-a617-3b51692e6db5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01199c800bed442bbb84de9d8d66eb50)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:42.421080 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406f74ad-995c-42f7-b36e-31ef881a7f6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:42.423041 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406f74ad-995c-42f7-b36e-31ef881a7f6e service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:42.423479 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda6eb2f-e173-4c63-a617-3b51692e6db5 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:42.438734 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 964/1924] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:38:42.533797 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-dda6eb2f-e173-4c63-a617-3b51692e6db5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01199c800bed442bbb84de9d8d66eb50', 'request_id': 'req-dda6eb2f-e173-4c63-a617-3b51692e6db5'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:42.546986 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-dda6eb2f-e173-4c63-a617-3b51692e6db5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01199c800bed442bbb84de9d8d66eb50', 'request_id': 'req-dda6eb2f-e173-4c63-a617-3b51692e6db5'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:42.556200 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-dda6eb2f-e173-4c63-a617-3b51692e6db5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01199c800bed442bbb84de9d8d66eb50', 'request_id': 'req-dda6eb2f-e173-4c63-a617-3b51692e6db5'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:42.569681 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-dda6eb2f-e173-4c63-a617-3b51692e6db5 admin admin] The token cache is being invalidate because project 01199c800bed442bbb84de9d8d66eb50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:42.570050 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-dda6eb2f-e173-4c63-a617-3b51692e6db5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:42.570658 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-dda6eb2f-e173-4c63-a617-3b51692e6db5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:42.572641 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 961/1925] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:42 2022] DELETE /identity/v3/projects/01199c800bed442bbb84de9d8d66eb50 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:42.578946 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b3908-98cb-4419-b192-ed5de65d6b35 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:42.579574 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b3908-98cb-4419-b192-ed5de65d6b35 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:42.580143 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b3908-98cb-4419-b192-ed5de65d6b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:42.609191 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-7d9b3908-98cb-4419-b192-ed5de65d6b35 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:42.610505 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-7d9b3908-98cb-4419-b192-ed5de65d6b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:42.625221 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:42.704357 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 965/1926] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:42.706306 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58c509-b538-4982-b04e-801718860e4f None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:42.706669 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58c509-b538-4982-b04e-801718860e4f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:42.707021 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58c509-b538-4982-b04e-801718860e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:42.729932 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-2c58c509-b538-4982-b04e-801718860e4f None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:42.731410 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-2c58c509-b538-4982-b04e-801718860e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:42.740280 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 962/1927] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:42.744846 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b3908-98cb-4419-b192-ed5de65d6b35 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:42.754697 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b3908-98cb-4419-b192-ed5de65d6b35 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:42.759303 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af2a70-b9e4-43c3-9382-0541f971d999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:42.759992 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af2a70-b9e4-43c3-9382-0541f971d999 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:42.760277 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af2a70-b9e4-43c3-9382-0541f971d999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:42.760555 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af2a70-b9e4-43c3-9382-0541f971d999 admin admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:42.761832 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20af2a70-b9e4-43c3-9382-0541f971d999 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:38:42.762162 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20af2a70-b9e4-43c3-9382-0541f971d999 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:42.763766 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20af2a70-b9e4-43c3-9382-0541f971d999 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:42.786141 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 966/1928] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:38:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:38:42.788448 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58c509-b538-4982-b04e-801718860e4f None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:42.798417 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58c509-b538-4982-b04e-801718860e4f None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:42.803399 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a1b340-4ec4-4b5b-8d6d-4d365f804b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:42.804122 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1b340-4ec4-4b5b-8d6d-4d365f804b8b admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:42.804465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1b340-4ec4-4b5b-8d6d-4d365f804b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:42.804816 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1b340-4ec4-4b5b-8d6d-4d365f804b8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:42.805586 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a1b340-4ec4-4b5b-8d6d-4d365f804b8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:42.807773 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a1b340-4ec4-4b5b-8d6d-4d365f804b8b admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:42.849025 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 963/1929] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:42 2022] POST /identity/v3/projects => generated 387 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:42.852893 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af2a70-b9e4-43c3-9382-0541f971d999 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:42.867088 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af2a70-b9e4-43c3-9382-0541f971d999 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:42.946901 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2986207d-5513-4b59-b221-9dbbc7ed8bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:42.947891 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2986207d-5513-4b59-b221-9dbbc7ed8bb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:42.949450 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2986207d-5513-4b59-b221-9dbbc7ed8bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:42.949996 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2986207d-5513-4b59-b221-9dbbc7ed8bb8 admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:42.950810 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2986207d-5513-4b59-b221-9dbbc7ed8bb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:42.953194 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2986207d-5513-4b59-b221-9dbbc7ed8bb8 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:43.037703 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a1b340-4ec4-4b5b-8d6d-4d365f804b8b admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:43.037893 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 967/1930] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:42 2022] POST /identity/v3/users => generated 294 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:43.039227 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-77a1b340-4ec4-4b5b-8d6d-4d365f804b8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:43.039861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2986207d-5513-4b59-b221-9dbbc7ed8bb8 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:43.052412 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2986207d-5513-4b59-b221-9dbbc7ed8bb8 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:43.057303 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df7e3d-7fac-4ed3-a251-b5993458d3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:43.058928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df7e3d-7fac-4ed3-a251-b5993458d3dd admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:43.059083 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df7e3d-7fac-4ed3-a251-b5993458d3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:43.059322 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df7e3d-7fac-4ed3-a251-b5993458d3dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:43.060031 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31df7e3d-7fac-4ed3-a251-b5993458d3dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:43.061737 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31df7e3d-7fac-4ed3-a251-b5993458d3dd admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:43.064756 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a1b340-4ec4-4b5b-8d6d-4d365f804b8b admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:43.077854 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 968/1931] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:43.081519 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df7e3d-7fac-4ed3-a251-b5993458d3dd admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:43.086780 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac29b40e-91e0-43e5-9a19-9c0c45ce5bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:43.088770 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29b40e-91e0-43e5-9a19-9c0c45ce5bc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:43.093127 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29b40e-91e0-43e5-9a19-9c0c45ce5bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:43.093496 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29b40e-91e0-43e5-9a19-9c0c45ce5bc2 admin admin] PATH_INFO: `/v3/projects/a352916bad2f4935beb3bd9979bfd61c` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:43.093797 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df7e3d-7fac-4ed3-a251-b5993458d3dd admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:43.099274 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1994e0-8451-4c39-ac39-b1d18473ea8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:43.099646 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac29b40e-91e0-43e5-9a19-9c0c45ce5bc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a352916bad2f4935beb3bd9979bfd61c)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:43.099988 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1994e0-8451-4c39-ac39-b1d18473ea8a admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:43.100277 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1994e0-8451-4c39-ac39-b1d18473ea8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:43.102391 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1994e0-8451-4c39-ac39-b1d18473ea8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:43.102391 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1994e0-8451-4c39-ac39-b1d18473ea8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:43.102934 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1994e0-8451-4c39-ac39-b1d18473ea8a admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:43.110560 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac29b40e-91e0-43e5-9a19-9c0c45ce5bc2 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:43.117426 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 969/1932] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:43.123078 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1994e0-8451-4c39-ac39-b1d18473ea8a admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:43.140274 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1994e0-8451-4c39-ac39-b1d18473ea8a admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:43.154585 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c51ba52-7500-4ffb-9e6e-8f412466c5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:43.154585 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51ba52-7500-4ffb-9e6e-8f412466c5d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:43.154585 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51ba52-7500-4ffb-9e6e-8f412466c5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:43.154585 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51ba52-7500-4ffb-9e6e-8f412466c5d2 admin admin] PATH_INFO: `/v3/projects/e201952a2363437f8fee3f1d4f8604d6/users/f504452c495f404c80f9cb38ad75d194/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:43.192140 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c51ba52-7500-4ffb-9e6e-8f412466c5d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e201952a2363437f8fee3f1d4f8604d6, user_id=f504452c495f404c80f9cb38ad75d194, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:43.197283 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c51ba52-7500-4ffb-9e6e-8f412466c5d2 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:43.208219 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ac29b40e-91e0-43e5-9a19-9c0c45ce5bc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a352916bad2f4935beb3bd9979bfd61c', 'request_id': 'req-ac29b40e-91e0-43e5-9a19-9c0c45ce5bc2'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:43.219341 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 970/1933] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:43 2022] PUT /identity/v3/projects/e201952a2363437f8fee3f1d4f8604d6/users/f504452c495f404c80f9cb38ad75d194/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:43.230858 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ac29b40e-91e0-43e5-9a19-9c0c45ce5bc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a352916bad2f4935beb3bd9979bfd61c', 'request_id': 'req-ac29b40e-91e0-43e5-9a19-9c0c45ce5bc2'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:43.233688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c990639-a1e4-4df7-9148-04ad9da86ba0 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:43.233972 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c990639-a1e4-4df7-9148-04ad9da86ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:43.234283 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c990639-a1e4-4df7-9148-04ad9da86ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:43.241470 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ac29b40e-91e0-43e5-9a19-9c0c45ce5bc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a352916bad2f4935beb3bd9979bfd61c', 'request_id': 'req-ac29b40e-91e0-43e5-9a19-9c0c45ce5bc2'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:43.255489 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ac29b40e-91e0-43e5-9a19-9c0c45ce5bc2 admin admin] The token cache is being invalidate because project a352916bad2f4935beb3bd9979bfd61c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:43.255744 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ac29b40e-91e0-43e5-9a19-9c0c45ce5bc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:43.256608 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ac29b40e-91e0-43e5-9a19-9c0c45ce5bc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:43.261478 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 964/1934] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:43 2022] DELETE /identity/v3/projects/a352916bad2f4935beb3bd9979bfd61c => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:38:43.262548 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac29b40e-91e0-43e5-9a19-9c0c45ce5bc2 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:43.263932 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ac29b40e-91e0-43e5-9a19-9c0c45ce5bc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:43.283069 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-9c990639-a1e4-4df7-9148-04ad9da86ba0 None None] MFA Rules not processed for user `f504452c495f404c80f9cb38ad75d194`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:43.284415 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-9c990639-a1e4-4df7-9148-04ad9da86ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:43.296622 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:43.346658 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:43.384198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 971/1935] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:43.390050 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c990639-a1e4-4df7-9148-04ad9da86ba0 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:43.391277 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-9c990639-a1e4-4df7-9148-04ad9da86ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:43.395833 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac29b40e-91e0-43e5-9a19-9c0c45ce5bc2 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:43.478010 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c990639-a1e4-4df7-9148-04ad9da86ba0 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:43.484883 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f8093a-81ec-4c0b-9217-12e7343eb6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:43.485574 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8093a-81ec-4c0b-9217-12e7343eb6d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:43.485830 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8093a-81ec-4c0b-9217-12e7343eb6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:43.486195 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8093a-81ec-4c0b-9217-12e7343eb6d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:43.487059 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f8093a-81ec-4c0b-9217-12e7343eb6d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:43.489333 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f8093a-81ec-4c0b-9217-12e7343eb6d4 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:43.496817 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da927542-2868-4426-93c3-9878a29ca528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:43.497839 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da927542-2868-4426-93c3-9878a29ca528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:43.498334 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da927542-2868-4426-93c3-9878a29ca528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:43.498674 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da927542-2868-4426-93c3-9878a29ca528 admin admin] PATH_INFO: `/v3/users/3dcb6816aef145409e264bbb5f526fc2` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:43.500182 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da927542-2868-4426-93c3-9878a29ca528 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dcb6816aef145409e264bbb5f526fc2)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:43.503306 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da927542-2868-4426-93c3-9878a29ca528 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:43.522138 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 972/1936] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:43 2022] POST /identity/v3/projects => generated 387 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:38:43.526250 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f8093a-81ec-4c0b-9217-12e7343eb6d4 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:43.598824 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-da927542-2868-4426-93c3-9878a29ca528 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dcb6816aef145409e264bbb5f526fc2'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:43.602161 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-da927542-2868-4426-93c3-9878a29ca528 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dcb6816aef145409e264bbb5f526fc2'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:43.602888 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-da927542-2868-4426-93c3-9878a29ca528 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dcb6816aef145409e264bbb5f526fc2'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:43.612235 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f8093a-81ec-4c0b-9217-12e7343eb6d4 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:43.620128 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb50a8-5901-494f-8975-44baeee5c23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:43.621190 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb50a8-5901-494f-8975-44baeee5c23b admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:43.622050 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb50a8-5901-494f-8975-44baeee5c23b admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:43.622050 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb50a8-5901-494f-8975-44baeee5c23b admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:43.622564 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bb50a8-5901-494f-8975-44baeee5c23b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:43.625204 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bb50a8-5901-494f-8975-44baeee5c23b admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:43.631947 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-da927542-2868-4426-93c3-9878a29ca528 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dcb6816aef145409e264bbb5f526fc2'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:43.641254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-da927542-2868-4426-93c3-9878a29ca528 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dcb6816aef145409e264bbb5f526fc2'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:43.662939 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 973/1937] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:43 2022] POST /identity/v3/users => generated 300 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:43.666410 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb50a8-5901-494f-8975-44baeee5c23b admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:43.670806 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-75bb50a8-5901-494f-8975-44baeee5c23b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:43.673389 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 965/1938] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:38:43 2022] DELETE /identity/v3/users/3dcb6816aef145409e264bbb5f526fc2 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:43.700737 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb50a8-5901-494f-8975-44baeee5c23b admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:43.809996 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:43.818478 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d2939-4f03-41b3-9d10-fb285dbd57cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:43.819474 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d2939-4f03-41b3-9d10-fb285dbd57cf admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:43.820047 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d2939-4f03-41b3-9d10-fb285dbd57cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:43.820398 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d2939-4f03-41b3-9d10-fb285dbd57cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:43.821517 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6d2939-4f03-41b3-9d10-fb285dbd57cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:43.823356 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6d2939-4f03-41b3-9d10-fb285dbd57cf admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:43.835350 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 974/1939] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:43.840647 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da927542-2868-4426-93c3-9878a29ca528 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:43.852384 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da927542-2868-4426-93c3-9878a29ca528 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:43.858749 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876fcecf-570c-4e8e-8630-364cbcd98541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:43.860867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876fcecf-570c-4e8e-8630-364cbcd98541 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:43.861385 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876fcecf-570c-4e8e-8630-364cbcd98541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:43.862452 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876fcecf-570c-4e8e-8630-364cbcd98541 admin admin] PATH_INFO: `/v3/projects/30643ec32ad7455bbf7fa923f605b153/users/39759a6e5ca64833bc432001f97ff2ff/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:43.883077 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876fcecf-570c-4e8e-8630-364cbcd98541 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30643ec32ad7455bbf7fa923f605b153, user_id=39759a6e5ca64833bc432001f97ff2ff, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:43.886803 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876fcecf-570c-4e8e-8630-364cbcd98541 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:43.910914 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 966/1940] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:43 2022] PUT /identity/v3/projects/30643ec32ad7455bbf7fa923f605b153/users/39759a6e5ca64833bc432001f97ff2ff/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:43.912154 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d2939-4f03-41b3-9d10-fb285dbd57cf admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:43.924881 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d2939-4f03-41b3-9d10-fb285dbd57cf admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:44.049457 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b674eaf-91ee-4c07-a25b-4019f5c92e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:44.050806 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b674eaf-91ee-4c07-a25b-4019f5c92e6a admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:44.050806 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b674eaf-91ee-4c07-a25b-4019f5c92e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:44.051045 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b674eaf-91ee-4c07-a25b-4019f5c92e6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:44.052065 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b674eaf-91ee-4c07-a25b-4019f5c92e6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:44.054117 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b674eaf-91ee-4c07-a25b-4019f5c92e6a admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:44.069157 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 975/1941] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:44.070603 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876fcecf-570c-4e8e-8630-364cbcd98541 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:44.088123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876fcecf-570c-4e8e-8630-364cbcd98541 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:44.093194 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0bab5e-89ab-462e-a6f0-281d657e029e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:44.094126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0bab5e-89ab-462e-a6f0-281d657e029e admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:44.094417 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0bab5e-89ab-462e-a6f0-281d657e029e admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:44.094745 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0bab5e-89ab-462e-a6f0-281d657e029e admin admin] PATH_INFO: `/v3/projects/30643ec32ad7455bbf7fa923f605b153/users/39759a6e5ca64833bc432001f97ff2ff/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:44.096766 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0bab5e-89ab-462e-a6f0-281d657e029e admin admin] RBAC: Authorizing `identity:create_grant(project_id=30643ec32ad7455bbf7fa923f605b153, user_id=39759a6e5ca64833bc432001f97ff2ff, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:44.099477 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0bab5e-89ab-462e-a6f0-281d657e029e admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:44.111041 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 967/1942] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:44 2022] PUT /identity/v3/projects/30643ec32ad7455bbf7fa923f605b153/users/39759a6e5ca64833bc432001f97ff2ff/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:44.121795 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd065d1a-3840-48cc-b5c9-fa5be28f345b None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:44.122113 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd065d1a-3840-48cc-b5c9-fa5be28f345b None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:44.122397 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd065d1a-3840-48cc-b5c9-fa5be28f345b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:44.151075 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-dd065d1a-3840-48cc-b5c9-fa5be28f345b None None] MFA Rules not processed for user `39759a6e5ca64833bc432001f97ff2ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:44.153117 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-dd065d1a-3840-48cc-b5c9-fa5be28f345b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:44.228416 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 976/1943] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:44 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:44.237455 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0bab5e-89ab-462e-a6f0-281d657e029e admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:44.239486 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ed0bab5e-89ab-462e-a6f0-281d657e029e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:44.278648 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:44.349447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0bab5e-89ab-462e-a6f0-281d657e029e admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:44.353352 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4304a5d-79c2-4516-b88c-c14f6770be6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:44.354913 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4304a5d-79c2-4516-b88c-c14f6770be6d service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:44.356073 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4304a5d-79c2-4516-b88c-c14f6770be6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:44.356485 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4304a5d-79c2-4516-b88c-c14f6770be6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:44.368489 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4304a5d-79c2-4516-b88c-c14f6770be6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:44.370133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4304a5d-79c2-4516-b88c-c14f6770be6d service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:44.392322 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 968/1944] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:44 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:38:44.508830 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd065d1a-3840-48cc-b5c9-fa5be28f345b None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:44.512795 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-dd065d1a-3840-48cc-b5c9-fa5be28f345b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:44.556114 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4304a5d-79c2-4516-b88c-c14f6770be6d service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:44.613223 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd065d1a-3840-48cc-b5c9-fa5be28f345b None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:44.619529 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3bcc62-35ef-4c18-87ba-8f6d532a1d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:44.621031 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3bcc62-35ef-4c18-87ba-8f6d532a1d5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:44.621386 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3bcc62-35ef-4c18-87ba-8f6d532a1d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:44.621736 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3bcc62-35ef-4c18-87ba-8f6d532a1d5c admin admin] PATH_INFO: `/v3/projects/39337f1f97074cff8beaeec245d62e91` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:44.623226 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3bcc62-35ef-4c18-87ba-8f6d532a1d5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=39337f1f97074cff8beaeec245d62e91)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:44.625670 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3bcc62-35ef-4c18-87ba-8f6d532a1d5c admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:44.640948 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4304a5d-79c2-4516-b88c-c14f6770be6d service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:44.644734 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:44.646418 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf12a6d9-3c73-4744-b83b-990e44243155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:44.647924 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12a6d9-3c73-4744-b83b-990e44243155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:44.648367 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12a6d9-3c73-4744-b83b-990e44243155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:44.648729 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12a6d9-3c73-4744-b83b-990e44243155 admin admin] PATH_INFO: `/v3/users/f504452c495f404c80f9cb38ad75d194` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:44.650635 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf12a6d9-3c73-4744-b83b-990e44243155 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f504452c495f404c80f9cb38ad75d194)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:44.652865 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf12a6d9-3c73-4744-b83b-990e44243155 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:44.682108 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-ad3bcc62-35ef-4c18-87ba-8f6d532a1d5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39337f1f97074cff8beaeec245d62e91', 'request_id': 'req-ad3bcc62-35ef-4c18-87ba-8f6d532a1d5c'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:44.690777 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-ad3bcc62-35ef-4c18-87ba-8f6d532a1d5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39337f1f97074cff8beaeec245d62e91', 'request_id': 'req-ad3bcc62-35ef-4c18-87ba-8f6d532a1d5c'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:44.698254 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-ad3bcc62-35ef-4c18-87ba-8f6d532a1d5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39337f1f97074cff8beaeec245d62e91', 'request_id': 'req-ad3bcc62-35ef-4c18-87ba-8f6d532a1d5c'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:44.710498 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-cf12a6d9-3c73-4744-b83b-990e44243155 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f504452c495f404c80f9cb38ad75d194'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:44.711251 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-cf12a6d9-3c73-4744-b83b-990e44243155 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f504452c495f404c80f9cb38ad75d194'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:44.711517 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-ad3bcc62-35ef-4c18-87ba-8f6d532a1d5c admin admin] The token cache is being invalidate because project 39337f1f97074cff8beaeec245d62e91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:44.711815 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-ad3bcc62-35ef-4c18-87ba-8f6d532a1d5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:44.712064 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-cf12a6d9-3c73-4744-b83b-990e44243155 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f504452c495f404c80f9cb38ad75d194'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:44.712473 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-ad3bcc62-35ef-4c18-87ba-8f6d532a1d5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:44.714735 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 977/1945] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:44 2022] DELETE /identity/v3/projects/39337f1f97074cff8beaeec245d62e91 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:44.720855 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7b249-6163-4c28-8378-17f9f99c637a None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:44.721524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7b249-6163-4c28-8378-17f9f99c637a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:44.722529 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7b249-6163-4c28-8378-17f9f99c637a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:44.728214 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-cf12a6d9-3c73-4744-b83b-990e44243155 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f504452c495f404c80f9cb38ad75d194'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:44.752988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-cf12a6d9-3c73-4744-b83b-990e44243155 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f504452c495f404c80f9cb38ad75d194'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:44.754831 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-a5f7b249-6163-4c28-8378-17f9f99c637a None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:44.754831 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-a5f7b249-6163-4c28-8378-17f9f99c637a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:44.764633 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 969/1946] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:38:44 2022] DELETE /identity/v3/users/f504452c495f404c80f9cb38ad75d194 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:44.769638 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8dfa3-400d-4128-b473-cab930f35cf1 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:44.769932 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8dfa3-400d-4128-b473-cab930f35cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:44.770249 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8dfa3-400d-4128-b473-cab930f35cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:44.792137 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c2a8dfa3-400d-4128-b473-cab930f35cf1 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:44.794781 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c2a8dfa3-400d-4128-b473-cab930f35cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:44.819253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 978/1947] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:38:44.826003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:44.826561 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61e7ca-19e5-4de9-81fb-1198cbbb8ac8 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:44.826853 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61e7ca-19e5-4de9-81fb-1198cbbb8ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:44.827582 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61e7ca-19e5-4de9-81fb-1198cbbb8ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:44.850453 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-0f61e7ca-19e5-4de9-81fb-1198cbbb8ac8 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:44.851664 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-0f61e7ca-19e5-4de9-81fb-1198cbbb8ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:44.861399 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 970/1948] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:38:44.871572 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a8dfa3-400d-4128-b473-cab930f35cf1 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:44.873745 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c2a8dfa3-400d-4128-b473-cab930f35cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:44.928515 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 979/1949] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:38:44.932681 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a8dfa3-400d-4128-b473-cab930f35cf1 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:44.932816 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f61e7ca-19e5-4de9-81fb-1198cbbb8ac8 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:44.937040 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fafddc-9d5d-4ab7-93d5-b4d0b41d95c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:44.937697 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fafddc-9d5d-4ab7-93d5-b4d0b41d95c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:44.937978 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fafddc-9d5d-4ab7-93d5-b4d0b41d95c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:44.938289 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fafddc-9d5d-4ab7-93d5-b4d0b41d95c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:44.943913 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f61e7ca-19e5-4de9-81fb-1198cbbb8ac8 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:44.948890 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a285cc-3279-43f3-bdd8-aeb384fb7201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:44.949635 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a285cc-3279-43f3-bdd8-aeb384fb7201 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:44.949983 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fafddc-9d5d-4ab7-93d5-b4d0b41d95c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:44.950122 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a285cc-3279-43f3-bdd8-aeb384fb7201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:44.950444 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a285cc-3279-43f3-bdd8-aeb384fb7201 admin admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:44.951411 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a285cc-3279-43f3-bdd8-aeb384fb7201 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:38:44.951518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fafddc-9d5d-4ab7-93d5-b4d0b41d95c8 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:44.951604 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a285cc-3279-43f3-bdd8-aeb384fb7201 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:44.953229 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a285cc-3279-43f3-bdd8-aeb384fb7201 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:44.962986 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 971/1950] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:38:44.973218 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 980/1951] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:38:44.974959 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fafddc-9d5d-4ab7-93d5-b4d0b41d95c8 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:44.986799 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fafddc-9d5d-4ab7-93d5-b4d0b41d95c8 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:44.992842 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c6bce-254f-4123-ac3a-9afe5474c1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:44.993682 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c6bce-254f-4123-ac3a-9afe5474c1fb admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:44.994014 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c6bce-254f-4123-ac3a-9afe5474c1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:44.994289 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c6bce-254f-4123-ac3a-9afe5474c1fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:44.995628 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544c6bce-254f-4123-ac3a-9afe5474c1fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:44.997544 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544c6bce-254f-4123-ac3a-9afe5474c1fb admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:45.039005 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a285cc-3279-43f3-bdd8-aeb384fb7201 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:45.050142 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 972/1952] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:44 2022] POST /identity/v3/projects => generated 387 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:45.059021 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a285cc-3279-43f3-bdd8-aeb384fb7201 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:45.159307 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:45.253440 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cbeb08-08a3-4c2d-87e3-2dd7f8cfcbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:45.255483 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbeb08-08a3-4c2d-87e3-2dd7f8cfcbbd admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:45.269083 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbeb08-08a3-4c2d-87e3-2dd7f8cfcbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:45.276758 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbeb08-08a3-4c2d-87e3-2dd7f8cfcbbd admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:45.278214 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cbeb08-08a3-4c2d-87e3-2dd7f8cfcbbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:45.286497 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cbeb08-08a3-4c2d-87e3-2dd7f8cfcbbd admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:45.332672 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c6bce-254f-4123-ac3a-9afe5474c1fb admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:45.332672 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-544c6bce-254f-4123-ac3a-9afe5474c1fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:45.360617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c6bce-254f-4123-ac3a-9afe5474c1fb admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:45.371291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b944d-965b-4280-9f62-2c205221eee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:45.372382 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b944d-965b-4280-9f62-2c205221eee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:45.372531 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b944d-965b-4280-9f62-2c205221eee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:45.372776 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b944d-965b-4280-9f62-2c205221eee5 admin admin] PATH_INFO: `/v3/projects/e201952a2363437f8fee3f1d4f8604d6` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:45.374355 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075b944d-965b-4280-9f62-2c205221eee5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e201952a2363437f8fee3f1d4f8604d6)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:45.377162 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075b944d-965b-4280-9f62-2c205221eee5 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:45.396813 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 981/1953] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:45 2022] POST /identity/v3/users => generated 294 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:45.401054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cbeb08-08a3-4c2d-87e3-2dd7f8cfcbbd admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:45.422278 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cbeb08-08a3-4c2d-87e3-2dd7f8cfcbbd admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:45.433938 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa6126a-dcda-468f-9cd4-e7731649a356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:45.434785 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6126a-dcda-468f-9cd4-e7731649a356 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:45.435075 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6126a-dcda-468f-9cd4-e7731649a356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:45.435352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6126a-dcda-468f-9cd4-e7731649a356 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:45.436108 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa6126a-dcda-468f-9cd4-e7731649a356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:45.437745 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa6126a-dcda-468f-9cd4-e7731649a356 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:45.463901 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 982/1954] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:38:45.465233 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-075b944d-965b-4280-9f62-2c205221eee5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e201952a2363437f8fee3f1d4f8604d6', 'request_id': 'req-075b944d-965b-4280-9f62-2c205221eee5'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:45.466547 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa6126a-dcda-468f-9cd4-e7731649a356 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:45.479635 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-075b944d-965b-4280-9f62-2c205221eee5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e201952a2363437f8fee3f1d4f8604d6', 'request_id': 'req-075b944d-965b-4280-9f62-2c205221eee5'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:45.489382 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-075b944d-965b-4280-9f62-2c205221eee5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e201952a2363437f8fee3f1d4f8604d6', 'request_id': 'req-075b944d-965b-4280-9f62-2c205221eee5'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:45.492580 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa6126a-dcda-468f-9cd4-e7731649a356 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:45.509341 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-075b944d-965b-4280-9f62-2c205221eee5 admin admin] The token cache is being invalidate because project e201952a2363437f8fee3f1d4f8604d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:45.509653 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-075b944d-965b-4280-9f62-2c205221eee5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:45.510350 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-075b944d-965b-4280-9f62-2c205221eee5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:45.515680 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 973/1955] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:45 2022] DELETE /identity/v3/projects/e201952a2363437f8fee3f1d4f8604d6 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:45.525799 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b944d-965b-4280-9f62-2c205221eee5 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:45.534687 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-075b944d-965b-4280-9f62-2c205221eee5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:45.620137 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1564fb5d-a504-4897-bd1f-5f9a3abc6956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:45.621313 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564fb5d-a504-4897-bd1f-5f9a3abc6956 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:45.621505 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564fb5d-a504-4897-bd1f-5f9a3abc6956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:45.621874 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564fb5d-a504-4897-bd1f-5f9a3abc6956 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:45.625132 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1564fb5d-a504-4897-bd1f-5f9a3abc6956 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:45.628349 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1564fb5d-a504-4897-bd1f-5f9a3abc6956 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:45.646901 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 983/1956] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:38:45.653977 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:45.659948 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1564fb5d-a504-4897-bd1f-5f9a3abc6956 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:45.667049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-1564fb5d-a504-4897-bd1f-5f9a3abc6956 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:45.701826 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1564fb5d-a504-4897-bd1f-5f9a3abc6956 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:45.702117 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b944d-965b-4280-9f62-2c205221eee5 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:45.713699 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b02cea4-d87f-4444-b4d5-f3656e56040d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:45.714494 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b02cea4-d87f-4444-b4d5-f3656e56040d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:45.714782 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b02cea4-d87f-4444-b4d5-f3656e56040d admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:45.715003 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea4075-8752-40d4-bc96-83616e8cf8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:45.715147 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b02cea4-d87f-4444-b4d5-f3656e56040d admin admin] PATH_INFO: `/v3/users/39759a6e5ca64833bc432001f97ff2ff` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:45.716805 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea4075-8752-40d4-bc96-83616e8cf8c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:45.717538 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea4075-8752-40d4-bc96-83616e8cf8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:45.718933 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b02cea4-d87f-4444-b4d5-f3656e56040d admin admin] RBAC: Authorizing `identity:delete_user(user_id=39759a6e5ca64833bc432001f97ff2ff)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:45.720049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea4075-8752-40d4-bc96-83616e8cf8c9 admin admin] PATH_INFO: `/v3/projects/12965e75dc4942078d5ae8469630742a/users/ecf11f8cb13146a7a70b5a4dcf45dfdb/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:45.720902 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b02cea4-d87f-4444-b4d5-f3656e56040d admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:45.762766 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ea4075-8752-40d4-bc96-83616e8cf8c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12965e75dc4942078d5ae8469630742a, user_id=ecf11f8cb13146a7a70b5a4dcf45dfdb, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:45.765599 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ea4075-8752-40d4-bc96-83616e8cf8c9 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:45.795501 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 984/1957] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:45 2022] PUT /identity/v3/projects/12965e75dc4942078d5ae8469630742a/users/ecf11f8cb13146a7a70b5a4dcf45dfdb/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:45.806731 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025f425-589a-4eee-a732-0542cc504ace None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:45.807025 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025f425-589a-4eee-a732-0542cc504ace None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:45.807304 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025f425-589a-4eee-a732-0542cc504ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:45.835747 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4b02cea4-d87f-4444-b4d5-f3656e56040d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39759a6e5ca64833bc432001f97ff2ff'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:45.836865 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4b02cea4-d87f-4444-b4d5-f3656e56040d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39759a6e5ca64833bc432001f97ff2ff'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:45.837748 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4b02cea4-d87f-4444-b4d5-f3656e56040d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39759a6e5ca64833bc432001f97ff2ff'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:45.842884 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-d025f425-589a-4eee-a732-0542cc504ace None None] MFA Rules not processed for user `ecf11f8cb13146a7a70b5a4dcf45dfdb`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:45.844230 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-d025f425-589a-4eee-a732-0542cc504ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:45.863081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4b02cea4-d87f-4444-b4d5-f3656e56040d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39759a6e5ca64833bc432001f97ff2ff'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:45.905939 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4b02cea4-d87f-4444-b4d5-f3656e56040d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39759a6e5ca64833bc432001f97ff2ff'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:45.924473 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 974/1958] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:38:45 2022] DELETE /identity/v3/users/39759a6e5ca64833bc432001f97ff2ff => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:45.940038 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 985/1959] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:45.944514 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b02cea4-d87f-4444-b4d5-f3656e56040d admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:45.946155 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-4b02cea4-d87f-4444-b4d5-f3656e56040d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:46.035945 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b02cea4-d87f-4444-b4d5-f3656e56040d admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:46.041674 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d61cb-0e2c-4214-8b8f-c32b8de08d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:46.043427 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d61cb-0e2c-4214-8b8f-c32b8de08d77 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:46.043959 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d61cb-0e2c-4214-8b8f-c32b8de08d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:46.044315 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d61cb-0e2c-4214-8b8f-c32b8de08d77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:46.045106 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d61cb-0e2c-4214-8b8f-c32b8de08d77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:46.047069 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d61cb-0e2c-4214-8b8f-c32b8de08d77 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:46.077510 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 975/1960] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:45 2022] POST /identity/v3/projects => generated 387 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:46.081202 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025f425-589a-4eee-a732-0542cc504ace None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:46.097538 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:46.180836 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025f425-589a-4eee-a732-0542cc504ace None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:46.271886 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8b7873-4054-4afe-b3e2-6bf40fc6c55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:46.272571 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b7873-4054-4afe-b3e2-6bf40fc6c55d admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:46.272853 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b7873-4054-4afe-b3e2-6bf40fc6c55d admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:46.273125 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b7873-4054-4afe-b3e2-6bf40fc6c55d admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:46.273824 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8b7873-4054-4afe-b3e2-6bf40fc6c55d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:46.276320 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8b7873-4054-4afe-b3e2-6bf40fc6c55d admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:46.307486 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 986/1961] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:46 2022] POST /identity/v3/users => generated 300 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:46.308913 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d61cb-0e2c-4214-8b8f-c32b8de08d77 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:46.320277 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d61cb-0e2c-4214-8b8f-c32b8de08d77 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:46.326728 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799cb52f-025f-4392-85d4-711b47d6d6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:46.328000 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cb52f-025f-4392-85d4-711b47d6d6be admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:46.328891 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cb52f-025f-4392-85d4-711b47d6d6be admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:46.329408 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cb52f-025f-4392-85d4-711b47d6d6be admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:46.330485 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799cb52f-025f-4392-85d4-711b47d6d6be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:46.332482 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799cb52f-025f-4392-85d4-711b47d6d6be admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:46.374358 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 976/1962] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:46 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:46.376297 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8b7873-4054-4afe-b3e2-6bf40fc6c55d admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:46.391914 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8b7873-4054-4afe-b3e2-6bf40fc6c55d admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:46.397544 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420817aa-0e01-4859-aa2a-b9d85a8c6a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:46.398484 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420817aa-0e01-4859-aa2a-b9d85a8c6a2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:46.398772 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420817aa-0e01-4859-aa2a-b9d85a8c6a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:46.399221 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420817aa-0e01-4859-aa2a-b9d85a8c6a2f admin admin] PATH_INFO: `/v3/projects/e98b145b94014099b75b534f5e454969/users/53dff930f4e94ec990d0443763fa9942/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:46.419033 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420817aa-0e01-4859-aa2a-b9d85a8c6a2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e98b145b94014099b75b534f5e454969, user_id=53dff930f4e94ec990d0443763fa9942, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:46.421590 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420817aa-0e01-4859-aa2a-b9d85a8c6a2f admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:46.441957 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 987/1963] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:46 2022] PUT /identity/v3/projects/e98b145b94014099b75b534f5e454969/users/53dff930f4e94ec990d0443763fa9942/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:46.444535 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799cb52f-025f-4392-85d4-711b47d6d6be admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:46.456564 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799cb52f-025f-4392-85d4-711b47d6d6be admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:46.490044 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:46.525351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90bc9c3-f11c-47bd-90dc-9ebf2608c6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:46.526228 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90bc9c3-f11c-47bd-90dc-9ebf2608c6b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:46.526673 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90bc9c3-f11c-47bd-90dc-9ebf2608c6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:46.528271 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90bc9c3-f11c-47bd-90dc-9ebf2608c6b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:46.528640 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90bc9c3-f11c-47bd-90dc-9ebf2608c6b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:46.530398 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90bc9c3-f11c-47bd-90dc-9ebf2608c6b7 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:46.541699 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 977/1964] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:46 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:46.546114 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420817aa-0e01-4859-aa2a-b9d85a8c6a2f admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:46.558817 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420817aa-0e01-4859-aa2a-b9d85a8c6a2f admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:46.564172 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8177417a-2179-426f-8e6e-1a9e702da0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:46.565078 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8177417a-2179-426f-8e6e-1a9e702da0ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:46.565358 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8177417a-2179-426f-8e6e-1a9e702da0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:46.565627 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8177417a-2179-426f-8e6e-1a9e702da0ed admin admin] PATH_INFO: `/v3/projects/e98b145b94014099b75b534f5e454969/users/53dff930f4e94ec990d0443763fa9942/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:46.567690 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8177417a-2179-426f-8e6e-1a9e702da0ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=e98b145b94014099b75b534f5e454969, user_id=53dff930f4e94ec990d0443763fa9942, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:46.570743 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8177417a-2179-426f-8e6e-1a9e702da0ed admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:46.612850 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 988/1965] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:46 2022] PUT /identity/v3/projects/e98b145b94014099b75b534f5e454969/users/53dff930f4e94ec990d0443763fa9942/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:46.623508 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68ed68-a307-438b-9f4f-d7f3f88f4166 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:46.623839 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68ed68-a307-438b-9f4f-d7f3f88f4166 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:46.624072 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68ed68-a307-438b-9f4f-d7f3f88f4166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:46.657677 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-2b68ed68-a307-438b-9f4f-d7f3f88f4166 None None] MFA Rules not processed for user `53dff930f4e94ec990d0443763fa9942`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:46.658959 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-2b68ed68-a307-438b-9f4f-d7f3f88f4166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:46.743957 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 978/1966] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:46 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:46.757396 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8177417a-2179-426f-8e6e-1a9e702da0ed admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:46.759316 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-8177417a-2179-426f-8e6e-1a9e702da0ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:46.899259 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8177417a-2179-426f-8e6e-1a9e702da0ed admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:46.903277 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65d9fb-4434-4284-a021-9fe44f4fec8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:46.904049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65d9fb-4434-4284-a021-9fe44f4fec8b service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:46.904395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65d9fb-4434-4284-a021-9fe44f4fec8b service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:46.904743 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65d9fb-4434-4284-a021-9fe44f4fec8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:46.926720 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65d9fb-4434-4284-a021-9fe44f4fec8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:46.926720 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65d9fb-4434-4284-a021-9fe44f4fec8b service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:46.937347 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68ed68-a307-438b-9f4f-d7f3f88f4166 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:46.939884 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-2b68ed68-a307-438b-9f4f-d7f3f88f4166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:46.941489 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 989/1967] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:46 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:38:46.979391 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68ed68-a307-438b-9f4f-d7f3f88f4166 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:47.004287 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc3103-81f0-46e3-a718-c3972b7b0499 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:47.005065 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc3103-81f0-46e3-a718-c3972b7b0499 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:47.005350 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc3103-81f0-46e3-a718-c3972b7b0499 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:47.008149 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc3103-81f0-46e3-a718-c3972b7b0499 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:47.010640 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c0dc3103-81f0-46e3-a718-c3972b7b0499 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:47.057160 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65d9fb-4434-4284-a021-9fe44f4fec8b service cinder] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:47.059682 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-cc65d9fb-4434-4284-a021-9fe44f4fec8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:47.092862 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:47.181592 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc3103-81f0-46e3-a718-c3972b7b0499 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:47.185160 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc3103-81f0-46e3-a718-c3972b7b0499 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:47.201516 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 979/1968] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:46 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:38:47.213004 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65d9fb-4434-4284-a021-9fe44f4fec8b service cinder] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:47.218559 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e7e69a-2f85-42ce-8c79-ba05c1f8aaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:47.219683 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7e69a-2f85-42ce-8c79-ba05c1f8aaed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:47.220291 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7e69a-2f85-42ce-8c79-ba05c1f8aaed admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:47.220718 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7e69a-2f85-42ce-8c79-ba05c1f8aaed admin admin] PATH_INFO: `/v3/projects/30643ec32ad7455bbf7fa923f605b153` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:47.222374 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e7e69a-2f85-42ce-8c79-ba05c1f8aaed admin admin] RBAC: Authorizing `identity:delete_project(project_id=30643ec32ad7455bbf7fa923f605b153)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:47.224834 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e7e69a-2f85-42ce-8c79-ba05c1f8aaed admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:47.324724 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-81e7e69a-2f85-42ce-8c79-ba05c1f8aaed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30643ec32ad7455bbf7fa923f605b153', 'request_id': 'req-81e7e69a-2f85-42ce-8c79-ba05c1f8aaed'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:47.345544 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-81e7e69a-2f85-42ce-8c79-ba05c1f8aaed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30643ec32ad7455bbf7fa923f605b153', 'request_id': 'req-81e7e69a-2f85-42ce-8c79-ba05c1f8aaed'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:47.355829 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-81e7e69a-2f85-42ce-8c79-ba05c1f8aaed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30643ec32ad7455bbf7fa923f605b153', 'request_id': 'req-81e7e69a-2f85-42ce-8c79-ba05c1f8aaed'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:47.371267 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-81e7e69a-2f85-42ce-8c79-ba05c1f8aaed admin admin] The token cache is being invalidate because project 30643ec32ad7455bbf7fa923f605b153 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:47.371563 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-81e7e69a-2f85-42ce-8c79-ba05c1f8aaed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:47.372404 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-81e7e69a-2f85-42ce-8c79-ba05c1f8aaed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:47.374966 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 990/1969] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:47 2022] DELETE /identity/v3/projects/30643ec32ad7455bbf7fa923f605b153 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:38:47.380910 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c18cbb-6aac-4d0d-9a28-f484810703da None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:47.381284 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c18cbb-6aac-4d0d-9a28-f484810703da None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:47.381780 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c18cbb-6aac-4d0d-9a28-f484810703da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:47.413343 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-07c18cbb-6aac-4d0d-9a28-f484810703da None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:47.415047 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-07c18cbb-6aac-4d0d-9a28-f484810703da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:47.498602 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:47.534033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 980/1970] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:47.543131 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ee88f-3274-4206-896f-f339e5c52502 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:47.543651 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ee88f-3274-4206-896f-f339e5c52502 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:47.544256 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ee88f-3274-4206-896f-f339e5c52502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:47.597149 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-b92ee88f-3274-4206-896f-f339e5c52502 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:47.599683 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-b92ee88f-3274-4206-896f-f339e5c52502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:47.622347 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 991/1971] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:47.626752 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c18cbb-6aac-4d0d-9a28-f484810703da None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:47.646500 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c18cbb-6aac-4d0d-9a28-f484810703da None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:47.660387 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b70d2-3672-4fa1-8de2-17dc064d2783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:47.661260 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b70d2-3672-4fa1-8de2-17dc064d2783 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:47.661672 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b70d2-3672-4fa1-8de2-17dc064d2783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:47.662022 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b70d2-3672-4fa1-8de2-17dc064d2783 admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:47.662896 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2b70d2-3672-4fa1-8de2-17dc064d2783 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:38:47.663291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2b70d2-3672-4fa1-8de2-17dc064d2783 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:47.665018 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2b70d2-3672-4fa1-8de2-17dc064d2783 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:47.687111 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 981/1972] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:38:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:38:47.694542 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ee88f-3274-4206-896f-f339e5c52502 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:47.718761 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ee88f-3274-4206-896f-f339e5c52502 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:47.740165 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd9217-bac8-4e2b-83ab-7f8b2aa9ae88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:47.740898 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd9217-bac8-4e2b-83ab-7f8b2aa9ae88 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:47.743555 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd9217-bac8-4e2b-83ab-7f8b2aa9ae88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:47.743868 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd9217-bac8-4e2b-83ab-7f8b2aa9ae88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:47.754916 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd9217-bac8-4e2b-83ab-7f8b2aa9ae88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:47.757124 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd9217-bac8-4e2b-83ab-7f8b2aa9ae88 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:47.833826 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b70d2-3672-4fa1-8de2-17dc064d2783 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:47.841267 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 992/1973] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:47 2022] POST /identity/v3/projects => generated 379 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:47.857107 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b70d2-3672-4fa1-8de2-17dc064d2783 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:47.958145 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd9217-bac8-4e2b-83ab-7f8b2aa9ae88 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:47.960229 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-11bd9217-bac8-4e2b-83ab-7f8b2aa9ae88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:47.966008 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff955db6-6bfa-4203-bd5e-f934a939fded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:47.966723 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff955db6-6bfa-4203-bd5e-f934a939fded admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:47.967006 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff955db6-6bfa-4203-bd5e-f934a939fded admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:47.967288 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff955db6-6bfa-4203-bd5e-f934a939fded admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:47.970200 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff955db6-6bfa-4203-bd5e-f934a939fded admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:47.972357 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff955db6-6bfa-4203-bd5e-f934a939fded admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:47.976306 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd9217-bac8-4e2b-83ab-7f8b2aa9ae88 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:47.982150 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4f15f-9b10-4eab-81a0-15064e7c8a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:47.983190 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4f15f-9b10-4eab-81a0-15064e7c8a50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:47.983482 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4f15f-9b10-4eab-81a0-15064e7c8a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:47.983768 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4f15f-9b10-4eab-81a0-15064e7c8a50 admin admin] PATH_INFO: `/v3/users/ecf11f8cb13146a7a70b5a4dcf45dfdb` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:47.985148 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4f15f-9b10-4eab-81a0-15064e7c8a50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecf11f8cb13146a7a70b5a4dcf45dfdb)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:47.987300 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4f15f-9b10-4eab-81a0-15064e7c8a50 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:48.000648 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 982/1974] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:47 2022] POST /identity/v3/users => generated 290 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:38:48.005147 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff955db6-6bfa-4203-bd5e-f934a939fded admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:48.020456 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff955db6-6bfa-4203-bd5e-f934a939fded admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:48.029620 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a4f351-6627-4881-83a2-5981f4e3d94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:48.030592 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4f351-6627-4881-83a2-5981f4e3d94c admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:48.030990 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4f351-6627-4881-83a2-5981f4e3d94c admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:48.031277 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4f351-6627-4881-83a2-5981f4e3d94c admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:48.032123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a4f351-6627-4881-83a2-5981f4e3d94c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:48.034497 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a4f351-6627-4881-83a2-5981f4e3d94c admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:48.056283 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 983/1975] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:48.060975 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a4f351-6627-4881-83a2-5981f4e3d94c admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:48.072149 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a4f351-6627-4881-83a2-5981f4e3d94c admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:48.077720 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-94d4f15f-9b10-4eab-81a0-15064e7c8a50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecf11f8cb13146a7a70b5a4dcf45dfdb'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:48.078584 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-94d4f15f-9b10-4eab-81a0-15064e7c8a50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecf11f8cb13146a7a70b5a4dcf45dfdb'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:48.079123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d59e9-c57f-4dc5-84d6-e66ae3f39834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:48.079968 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d59e9-c57f-4dc5-84d6-e66ae3f39834 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:48.081266 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d59e9-c57f-4dc5-84d6-e66ae3f39834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:48.081847 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d59e9-c57f-4dc5-84d6-e66ae3f39834 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:48.083226 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08d59e9-c57f-4dc5-84d6-e66ae3f39834 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:48.086328 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08d59e9-c57f-4dc5-84d6-e66ae3f39834 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:48.099325 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-94d4f15f-9b10-4eab-81a0-15064e7c8a50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecf11f8cb13146a7a70b5a4dcf45dfdb'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:48.100275 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 984/1976] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:38:48.102757 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-94d4f15f-9b10-4eab-81a0-15064e7c8a50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecf11f8cb13146a7a70b5a4dcf45dfdb'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:48.106254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d59e9-c57f-4dc5-84d6-e66ae3f39834 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:48.108616 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f08d59e9-c57f-4dc5-84d6-e66ae3f39834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:48.114980 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-94d4f15f-9b10-4eab-81a0-15064e7c8a50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecf11f8cb13146a7a70b5a4dcf45dfdb'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:48.122556 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:48.127143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d59e9-c57f-4dc5-84d6-e66ae3f39834 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:48.135053 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33000d3-371a-46c6-b421-10cac727970c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:48.136875 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33000d3-371a-46c6-b421-10cac727970c admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:48.137231 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33000d3-371a-46c6-b421-10cac727970c admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:48.137581 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33000d3-371a-46c6-b421-10cac727970c admin admin] PATH_INFO: `/v3/projects/3480b0308f0f44fa84a0ae21f966d71a/users/d108703b84904b04b52ac17892b30a75/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:48.141723 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 993/1977] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:38:47 2022] DELETE /identity/v3/users/ecf11f8cb13146a7a70b5a4dcf45dfdb => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:48.146201 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80993e01-34e6-4e51-a3aa-37ec9a639e1c None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:48.146684 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80993e01-34e6-4e51-a3aa-37ec9a639e1c None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:48.147265 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80993e01-34e6-4e51-a3aa-37ec9a639e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:48.157982 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33000d3-371a-46c6-b421-10cac727970c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3480b0308f0f44fa84a0ae21f966d71a, user_id=d108703b84904b04b52ac17892b30a75, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:48.160995 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33000d3-371a-46c6-b421-10cac727970c admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:48.172661 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-80993e01-34e6-4e51-a3aa-37ec9a639e1c None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:48.175270 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-80993e01-34e6-4e51-a3aa-37ec9a639e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:48.178129 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 985/1978] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:48 2022] PUT /identity/v3/projects/3480b0308f0f44fa84a0ae21f966d71a/users/d108703b84904b04b52ac17892b30a75/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:38:48.188817 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47455e6d-683a-4671-9257-f0004501260f None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:48.190124 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47455e6d-683a-4671-9257-f0004501260f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:48.190124 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47455e6d-683a-4671-9257-f0004501260f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:48.220560 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-47455e6d-683a-4671-9257-f0004501260f None None] MFA Rules not processed for user `d108703b84904b04b52ac17892b30a75`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:48.221378 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-47455e6d-683a-4671-9257-f0004501260f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:48.247469 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 994/1979] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:38:48.258916 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80993e01-34e6-4e51-a3aa-37ec9a639e1c None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:48.261515 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-80993e01-34e6-4e51-a3aa-37ec9a639e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:48.305914 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 986/1980] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:38:48.311666 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47455e6d-683a-4671-9257-f0004501260f None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:48.332226 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47455e6d-683a-4671-9257-f0004501260f None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:48.343422 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b3665-ba86-42c5-9611-198ae344a90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:48.348349 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b3665-ba86-42c5-9611-198ae344a90b admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:48.349127 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b3665-ba86-42c5-9611-198ae344a90b admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:48.349831 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b3665-ba86-42c5-9611-198ae344a90b admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:48.350743 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815b3665-ba86-42c5-9611-198ae344a90b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:48.353576 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815b3665-ba86-42c5-9611-198ae344a90b admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:48.357459 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:48.373193 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80993e01-34e6-4e51-a3aa-37ec9a639e1c None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:48.378636 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e03c4-6273-457b-8443-ca59ad39ec7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:48.379425 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e03c4-6273-457b-8443-ca59ad39ec7d service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:48.379576 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e03c4-6273-457b-8443-ca59ad39ec7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:48.379850 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e03c4-6273-457b-8443-ca59ad39ec7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:48.393088 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 987/1981] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:48 2022] POST /identity/v3/projects => generated 377 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:38:48.396962 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b3665-ba86-42c5-9611-198ae344a90b admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:48.398057 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667e03c4-6273-457b-8443-ca59ad39ec7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:48.400036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667e03c4-6273-457b-8443-ca59ad39ec7d service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:48.413731 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 995/1982] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:38:48.504783 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b3665-ba86-42c5-9611-198ae344a90b admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:48.510767 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9527c170-6c3f-4e04-8f03-771fdddc970f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:48.512009 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9527c170-6c3f-4e04-8f03-771fdddc970f admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:48.512522 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9527c170-6c3f-4e04-8f03-771fdddc970f admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:48.512996 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9527c170-6c3f-4e04-8f03-771fdddc970f admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:48.513893 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9527c170-6c3f-4e04-8f03-771fdddc970f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:48.517026 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9527c170-6c3f-4e04-8f03-771fdddc970f admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:48.540622 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 988/1983] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:48 2022] POST /identity/v3/users => generated 289 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:48.545554 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e03c4-6273-457b-8443-ca59ad39ec7d service neutron] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:48.556715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e03c4-6273-457b-8443-ca59ad39ec7d service neutron] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:48.562154 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d8b130-c7d3-46d0-b489-f82438d53840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:48.562894 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d8b130-c7d3-46d0-b489-f82438d53840 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:48.563253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d8b130-c7d3-46d0-b489-f82438d53840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:48.563596 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d8b130-c7d3-46d0-b489-f82438d53840 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:48.564333 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d8b130-c7d3-46d0-b489-f82438d53840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:48.566073 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d8b130-c7d3-46d0-b489-f82438d53840 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:48.578132 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 996/1984] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:48.582563 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9527c170-6c3f-4e04-8f03-771fdddc970f admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:48.593718 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9527c170-6c3f-4e04-8f03-771fdddc970f admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:48.598686 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a62002b-5038-4604-9bba-3e996e66e7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:48.599357 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62002b-5038-4604-9bba-3e996e66e7d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:48.599640 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62002b-5038-4604-9bba-3e996e66e7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:48.599917 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62002b-5038-4604-9bba-3e996e66e7d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:48.600701 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a62002b-5038-4604-9bba-3e996e66e7d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:48.602345 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a62002b-5038-4604-9bba-3e996e66e7d6 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:48.613959 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 989/1985] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:38:48.618594 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d8b130-c7d3-46d0-b489-f82438d53840 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:48.656430 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d8b130-c7d3-46d0-b489-f82438d53840 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:48.677945 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0020f66-fb8a-4d7e-bc22-1df170694861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:48.678929 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0020f66-fb8a-4d7e-bc22-1df170694861 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:48.679498 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0020f66-fb8a-4d7e-bc22-1df170694861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:48.679498 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0020f66-fb8a-4d7e-bc22-1df170694861 admin admin] PATH_INFO: `/v3/projects/3b247a30aa2841a698be29e5aaf92edc/users/e5bf6924564746dc98c77adf6c5904f0/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:48.716364 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0020f66-fb8a-4d7e-bc22-1df170694861 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b247a30aa2841a698be29e5aaf92edc, user_id=e5bf6924564746dc98c77adf6c5904f0, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:48.719119 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0020f66-fb8a-4d7e-bc22-1df170694861 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:48.735446 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 997/1986] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:48 2022] PUT /identity/v3/projects/3b247a30aa2841a698be29e5aaf92edc/users/e5bf6924564746dc98c77adf6c5904f0/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:48.745578 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ff7d39-379d-4c95-b0fc-b5495c30da45 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:48.745917 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ff7d39-379d-4c95-b0fc-b5495c30da45 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:48.746791 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ff7d39-379d-4c95-b0fc-b5495c30da45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:48.776201 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-04ff7d39-379d-4c95-b0fc-b5495c30da45 None None] MFA Rules not processed for user `e5bf6924564746dc98c77adf6c5904f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:48.778404 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-04ff7d39-379d-4c95-b0fc-b5495c30da45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:48.915294 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0020f66-fb8a-4d7e-bc22-1df170694861 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:48.918220 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-f0020f66-fb8a-4d7e-bc22-1df170694861 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:48.984003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 990/1987] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:48.985119 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ff7d39-379d-4c95-b0fc-b5495c30da45 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:49.057648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:49.083975 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0020f66-fb8a-4d7e-bc22-1df170694861 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:49.099737 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24332c93-bb17-4ddd-9ed3-2605038a4abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:49.100928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24332c93-bb17-4ddd-9ed3-2605038a4abe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:49.101221 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24332c93-bb17-4ddd-9ed3-2605038a4abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:49.101498 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24332c93-bb17-4ddd-9ed3-2605038a4abe admin admin] PATH_INFO: `/v3/projects/12965e75dc4942078d5ae8469630742a` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:49.105216 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24332c93-bb17-4ddd-9ed3-2605038a4abe admin admin] RBAC: Authorizing `identity:delete_project(project_id=12965e75dc4942078d5ae8469630742a)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:49.107194 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24332c93-bb17-4ddd-9ed3-2605038a4abe admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:49.141805 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:49.176651 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ff7d39-379d-4c95-b0fc-b5495c30da45 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:49.194500 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca13ed-55c8-4c21-90fb-083c99f98262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:49.195524 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca13ed-55c8-4c21-90fb-083c99f98262 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:49.195807 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca13ed-55c8-4c21-90fb-083c99f98262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:49.196579 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca13ed-55c8-4c21-90fb-083c99f98262 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:49.200446 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca13ed-55c8-4c21-90fb-083c99f98262 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:49.205173 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca13ed-55c8-4c21-90fb-083c99f98262 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:49.220992 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-24332c93-bb17-4ddd-9ed3-2605038a4abe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12965e75dc4942078d5ae8469630742a', 'request_id': 'req-24332c93-bb17-4ddd-9ed3-2605038a4abe'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:49.236252 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-24332c93-bb17-4ddd-9ed3-2605038a4abe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12965e75dc4942078d5ae8469630742a', 'request_id': 'req-24332c93-bb17-4ddd-9ed3-2605038a4abe'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:49.247296 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-24332c93-bb17-4ddd-9ed3-2605038a4abe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12965e75dc4942078d5ae8469630742a', 'request_id': 'req-24332c93-bb17-4ddd-9ed3-2605038a4abe'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:49.267912 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-24332c93-bb17-4ddd-9ed3-2605038a4abe admin admin] The token cache is being invalidate because project 12965e75dc4942078d5ae8469630742a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:49.270159 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-24332c93-bb17-4ddd-9ed3-2605038a4abe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:49.271786 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-24332c93-bb17-4ddd-9ed3-2605038a4abe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:49.274063 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 991/1988] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:48 2022] POST /identity/v3/projects => generated 377 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:49.274689 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 998/1989] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:48 2022] DELETE /identity/v3/projects/12965e75dc4942078d5ae8469630742a => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:38:49.279894 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca13ed-55c8-4c21-90fb-083c99f98262 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:49.280470 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24332c93-bb17-4ddd-9ed3-2605038a4abe admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:49.282573 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-24332c93-bb17-4ddd-9ed3-2605038a4abe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:49.283725 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b4ca13ed-55c8-4c21-90fb-083c99f98262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:49.397120 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca13ed-55c8-4c21-90fb-083c99f98262 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:49.407357 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f438d-8dab-408c-b2ca-c9edd0833280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:49.408690 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f438d-8dab-408c-b2ca-c9edd0833280 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:49.408975 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f438d-8dab-408c-b2ca-c9edd0833280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:49.409389 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f438d-8dab-408c-b2ca-c9edd0833280 admin admin] PATH_INFO: `/v3/users/53dff930f4e94ec990d0443763fa9942` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:49.410597 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81f438d-8dab-408c-b2ca-c9edd0833280 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53dff930f4e94ec990d0443763fa9942)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:49.412574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81f438d-8dab-408c-b2ca-c9edd0833280 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:49.424631 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24332c93-bb17-4ddd-9ed3-2605038a4abe admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:49.432052 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f47217-6e7f-48c8-a65a-3f89e4428976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:49.433636 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f47217-6e7f-48c8-a65a-3f89e4428976 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:49.434878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f47217-6e7f-48c8-a65a-3f89e4428976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:49.435304 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f47217-6e7f-48c8-a65a-3f89e4428976 admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:49.436772 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f47217-6e7f-48c8-a65a-3f89e4428976 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:49.440310 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f47217-6e7f-48c8-a65a-3f89e4428976 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:49.476221 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 999/1990] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:49 2022] POST /identity/v3/users => generated 295 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:49.481394 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f47217-6e7f-48c8-a65a-3f89e4428976 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:49.486500 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f81f438d-8dab-408c-b2ca-c9edd0833280 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53dff930f4e94ec990d0443763fa9942'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:49.488109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f81f438d-8dab-408c-b2ca-c9edd0833280 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53dff930f4e94ec990d0443763fa9942'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:49.488880 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f81f438d-8dab-408c-b2ca-c9edd0833280 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53dff930f4e94ec990d0443763fa9942'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:49.504763 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f47217-6e7f-48c8-a65a-3f89e4428976 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:49.511913 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f81f438d-8dab-408c-b2ca-c9edd0833280 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53dff930f4e94ec990d0443763fa9942'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:49.512131 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420249c-0d2e-44c5-9996-598cea492cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:49.512737 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420249c-0d2e-44c5-9996-598cea492cf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:49.513275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420249c-0d2e-44c5-9996-598cea492cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:49.513555 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420249c-0d2e-44c5-9996-598cea492cf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:49.514320 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8420249c-0d2e-44c5-9996-598cea492cf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:49.515988 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8420249c-0d2e-44c5-9996-598cea492cf8 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:49.525293 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f81f438d-8dab-408c-b2ca-c9edd0833280 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53dff930f4e94ec990d0443763fa9942'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:49.530753 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1000/1991] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:49.537351 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420249c-0d2e-44c5-9996-598cea492cf8 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:49.542923 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-8420249c-0d2e-44c5-9996-598cea492cf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:49.559964 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420249c-0d2e-44c5-9996-598cea492cf8 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:49.566370 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 992/1992] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:38:49 2022] DELETE /identity/v3/users/53dff930f4e94ec990d0443763fa9942 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:49.635484 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02fc41-4773-4cdc-91cd-fc565ff90adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:49.636709 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02fc41-4773-4cdc-91cd-fc565ff90adf admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:49.637006 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02fc41-4773-4cdc-91cd-fc565ff90adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:49.637761 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02fc41-4773-4cdc-91cd-fc565ff90adf admin admin] PATH_INFO: `/v3/projects/04fb2007c7df4df39fded5d039b478c9/users/c9fb7e763a3445bcb2293e447548d8f4/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:49.649538 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:49.669453 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02fc41-4773-4cdc-91cd-fc565ff90adf admin admin] RBAC: Authorizing `identity:create_grant(project_id=04fb2007c7df4df39fded5d039b478c9, user_id=c9fb7e763a3445bcb2293e447548d8f4, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:49.673394 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02fc41-4773-4cdc-91cd-fc565ff90adf admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:49.697366 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1001/1993] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:49 2022] PUT /identity/v3/projects/04fb2007c7df4df39fded5d039b478c9/users/c9fb7e763a3445bcb2293e447548d8f4/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:49.701845 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f438d-8dab-408c-b2ca-c9edd0833280 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:49.718609 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f438d-8dab-408c-b2ca-c9edd0833280 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:49.811624 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:49.822931 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e408b32-b215-4fd7-8cbf-bc6655beae73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:49.823702 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e408b32-b215-4fd7-8cbf-bc6655beae73 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:49.824216 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e408b32-b215-4fd7-8cbf-bc6655beae73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:49.824625 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e408b32-b215-4fd7-8cbf-bc6655beae73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:49.825709 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e408b32-b215-4fd7-8cbf-bc6655beae73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:49.827573 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e408b32-b215-4fd7-8cbf-bc6655beae73 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:49.843973 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 993/1994] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:49.845090 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02fc41-4773-4cdc-91cd-fc565ff90adf admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:49.856720 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02fc41-4773-4cdc-91cd-fc565ff90adf admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:49.861851 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674d449-d657-4d06-b178-e3db8ca4bbc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:49.862895 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674d449-d657-4d06-b178-e3db8ca4bbc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:49.863280 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674d449-d657-4d06-b178-e3db8ca4bbc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:49.863454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674d449-d657-4d06-b178-e3db8ca4bbc5 admin admin] PATH_INFO: `/v3/projects/04fb2007c7df4df39fded5d039b478c9/users/c9fb7e763a3445bcb2293e447548d8f4/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:49.865513 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b674d449-d657-4d06-b178-e3db8ca4bbc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04fb2007c7df4df39fded5d039b478c9, user_id=c9fb7e763a3445bcb2293e447548d8f4, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:49.868727 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b674d449-d657-4d06-b178-e3db8ca4bbc5 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:49.882209 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1002/1995] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:49 2022] PUT /identity/v3/projects/04fb2007c7df4df39fded5d039b478c9/users/c9fb7e763a3445bcb2293e447548d8f4/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:49.893462 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1dd88-fa6d-41f4-963f-df548ca7b7df None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:49.893883 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1dd88-fa6d-41f4-963f-df548ca7b7df None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:49.894266 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1dd88-fa6d-41f4-963f-df548ca7b7df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:49.938991 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-18f1dd88-fa6d-41f4-963f-df548ca7b7df None None] MFA Rules not processed for user `c9fb7e763a3445bcb2293e447548d8f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:49.940255 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-18f1dd88-fa6d-41f4-963f-df548ca7b7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:49.958324 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674d449-d657-4d06-b178-e3db8ca4bbc5 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:49.965179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-b674d449-d657-4d06-b178-e3db8ca4bbc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:50.050826 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674d449-d657-4d06-b178-e3db8ca4bbc5 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:50.057453 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b2ccf1-7351-47cf-a73d-4e1b72465902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:50.059296 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2ccf1-7351-47cf-a73d-4e1b72465902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:50.061010 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2ccf1-7351-47cf-a73d-4e1b72465902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:50.061404 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2ccf1-7351-47cf-a73d-4e1b72465902 admin admin] PATH_INFO: `/v3/projects/e98b145b94014099b75b534f5e454969` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:50.062897 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b2ccf1-7351-47cf-a73d-4e1b72465902 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e98b145b94014099b75b534f5e454969)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:50.066204 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b2ccf1-7351-47cf-a73d-4e1b72465902 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:50.071247 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 994/1996] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:49 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:50.086950 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1dd88-fa6d-41f4-963f-df548ca7b7df None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:50.090198 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-18f1dd88-fa6d-41f4-963f-df548ca7b7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:50.161533 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-f5b2ccf1-7351-47cf-a73d-4e1b72465902 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e98b145b94014099b75b534f5e454969', 'request_id': 'req-f5b2ccf1-7351-47cf-a73d-4e1b72465902'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:50.180244 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-f5b2ccf1-7351-47cf-a73d-4e1b72465902 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e98b145b94014099b75b534f5e454969', 'request_id': 'req-f5b2ccf1-7351-47cf-a73d-4e1b72465902'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:50.189100 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-f5b2ccf1-7351-47cf-a73d-4e1b72465902 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e98b145b94014099b75b534f5e454969', 'request_id': 'req-f5b2ccf1-7351-47cf-a73d-4e1b72465902'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:50.195869 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1dd88-fa6d-41f4-963f-df548ca7b7df None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:50.217225 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-f5b2ccf1-7351-47cf-a73d-4e1b72465902 admin admin] The token cache is being invalidate because project e98b145b94014099b75b534f5e454969 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:50.218322 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-f5b2ccf1-7351-47cf-a73d-4e1b72465902 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:50.219801 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-f5b2ccf1-7351-47cf-a73d-4e1b72465902 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:50.224696 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1003/1997] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:49 2022] DELETE /identity/v3/projects/e98b145b94014099b75b534f5e454969 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:50.236496 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a83ea-5c30-4514-b4a5-daafec137984 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:50.244089 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a83ea-5c30-4514-b4a5-daafec137984 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:50.244527 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a83ea-5c30-4514-b4a5-daafec137984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:50.293730 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-977a83ea-5c30-4514-b4a5-daafec137984 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:50.294957 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-977a83ea-5c30-4514-b4a5-daafec137984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:50.295781 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1cc77-6f4c-4605-a607-080a2c0600ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:50.296724 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1cc77-6f4c-4605-a607-080a2c0600ed service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:50.296993 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1cc77-6f4c-4605-a607-080a2c0600ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:50.297373 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1cc77-6f4c-4605-a607-080a2c0600ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:50.299572 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-89c1cc77-6f4c-4605-a607-080a2c0600ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:50.367413 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:50.398219 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c1cc77-6f4c-4605-a607-080a2c0600ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:50.400100 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c1cc77-6f4c-4605-a607-080a2c0600ed service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:50.406814 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1004/1998] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:50.418472 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6a4fd-b8f8-4ddd-9680-ccf58af068b5 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:50.427065 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6a4fd-b8f8-4ddd-9680-ccf58af068b5 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:50.427456 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6a4fd-b8f8-4ddd-9680-ccf58af068b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:50.430339 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 995/1999] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:50 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:38:50.438620 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:50.459919 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-c5d6a4fd-b8f8-4ddd-9680-ccf58af068b5 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:50.462644 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-c5d6a4fd-b8f8-4ddd-9680-ccf58af068b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:50.484695 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1005/2000] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:50.486255 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1cc77-6f4c-4605-a607-080a2c0600ed service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:50.504006 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1cc77-6f4c-4605-a607-080a2c0600ed service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:50.522896 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e8afcd-84c7-4803-af81-ba3e07f8b36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:50.523784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e8afcd-84c7-4803-af81-ba3e07f8b36e admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:50.524022 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d6a4fd-b8f8-4ddd-9680-ccf58af068b5 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:50.524317 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e8afcd-84c7-4803-af81-ba3e07f8b36e admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:50.524594 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e8afcd-84c7-4803-af81-ba3e07f8b36e admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:50.525286 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-c5d6a4fd-b8f8-4ddd-9680-ccf58af068b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:50.525398 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e8afcd-84c7-4803-af81-ba3e07f8b36e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:38:50.525759 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e8afcd-84c7-4803-af81-ba3e07f8b36e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:50.528749 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e8afcd-84c7-4803-af81-ba3e07f8b36e admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:50.540865 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d6a4fd-b8f8-4ddd-9680-ccf58af068b5 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:50.546276 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb2a3ed-4d6f-4fb8-9032-5876584d662c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:50.548491 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 996/2001] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:38:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:38:50.548623 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2a3ed-4d6f-4fb8-9032-5876584d662c service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:50.548623 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2a3ed-4d6f-4fb8-9032-5876584d662c service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:50.548623 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2a3ed-4d6f-4fb8-9032-5876584d662c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:50.549968 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-6cb2a3ed-4d6f-4fb8-9032-5876584d662c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:50.551652 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e8afcd-84c7-4803-af81-ba3e07f8b36e admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:50.565753 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e8afcd-84c7-4803-af81-ba3e07f8b36e admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:50.572018 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c8a7c-5a05-41ea-8981-a4429e7bdad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:50.572893 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c8a7c-5a05-41ea-8981-a4429e7bdad5 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:50.573179 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c8a7c-5a05-41ea-8981-a4429e7bdad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:50.574665 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c8a7c-5a05-41ea-8981-a4429e7bdad5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:50.575383 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c8a7c-5a05-41ea-8981-a4429e7bdad5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:50.579394 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c8a7c-5a05-41ea-8981-a4429e7bdad5 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:50.621713 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 997/2002] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:50 2022] POST /identity/v3/projects => generated 381 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:50.623079 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c8a7c-5a05-41ea-8981-a4429e7bdad5 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:50.634281 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb2a3ed-4d6f-4fb8-9032-5876584d662c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:50.636586 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb2a3ed-4d6f-4fb8-9032-5876584d662c service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:50.647338 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1006/2003] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:50 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:38:50.699757 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c8a7c-5a05-41ea-8981-a4429e7bdad5 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:50.705128 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee1d89e-f057-46c7-b18b-aeafae10a81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:50.705812 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee1d89e-f057-46c7-b18b-aeafae10a81c admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:50.706082 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee1d89e-f057-46c7-b18b-aeafae10a81c admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:50.706415 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee1d89e-f057-46c7-b18b-aeafae10a81c admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:50.707148 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee1d89e-f057-46c7-b18b-aeafae10a81c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:50.709488 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee1d89e-f057-46c7-b18b-aeafae10a81c admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:50.735896 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 998/2004] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:50 2022] POST /identity/v3/users => generated 291 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:50.740240 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb2a3ed-4d6f-4fb8-9032-5876584d662c service cinder] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:50.752243 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb2a3ed-4d6f-4fb8-9032-5876584d662c service cinder] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:50.757065 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720dcc1d-80d8-40fc-8510-2480899b1fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:50.757746 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720dcc1d-80d8-40fc-8510-2480899b1fd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:50.758019 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720dcc1d-80d8-40fc-8510-2480899b1fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:50.758328 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720dcc1d-80d8-40fc-8510-2480899b1fd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:50.759061 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720dcc1d-80d8-40fc-8510-2480899b1fd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:50.760692 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720dcc1d-80d8-40fc-8510-2480899b1fd4 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:50.771392 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1007/2005] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:50.775725 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee1d89e-f057-46c7-b18b-aeafae10a81c admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:50.786542 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee1d89e-f057-46c7-b18b-aeafae10a81c admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:50.791654 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f728f71-8aba-45d1-8d88-3853d373b42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:50.793104 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f728f71-8aba-45d1-8d88-3853d373b42a admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:50.793422 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f728f71-8aba-45d1-8d88-3853d373b42a admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:50.793702 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f728f71-8aba-45d1-8d88-3853d373b42a admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:50.794513 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f728f71-8aba-45d1-8d88-3853d373b42a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:50.796539 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f728f71-8aba-45d1-8d88-3853d373b42a admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:50.813009 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 999/2006] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:50.814801 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720dcc1d-80d8-40fc-8510-2480899b1fd4 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:50.830405 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720dcc1d-80d8-40fc-8510-2480899b1fd4 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:50.836765 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc2e36-5741-4a97-ac2c-bd7d7141e4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:50.837656 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc2e36-5741-4a97-ac2c-bd7d7141e4c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:50.837945 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc2e36-5741-4a97-ac2c-bd7d7141e4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:50.838260 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc2e36-5741-4a97-ac2c-bd7d7141e4c1 admin admin] PATH_INFO: `/v3/projects/b7fa1b5b797542f7a65a56df53b0c5d8/users/2bb84dc148d9498787752dea558edef0/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:50.856313 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dc2e36-5741-4a97-ac2c-bd7d7141e4c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7fa1b5b797542f7a65a56df53b0c5d8, user_id=2bb84dc148d9498787752dea558edef0, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:50.859302 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dc2e36-5741-4a97-ac2c-bd7d7141e4c1 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:50.874985 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1008/2007] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:50 2022] PUT /identity/v3/projects/b7fa1b5b797542f7a65a56df53b0c5d8/users/2bb84dc148d9498787752dea558edef0/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:38:50.886744 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c017b3-7302-411e-9997-831055829190 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:50.887108 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c017b3-7302-411e-9997-831055829190 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:50.887376 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c017b3-7302-411e-9997-831055829190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:50.914114 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-e4c017b3-7302-411e-9997-831055829190 None None] MFA Rules not processed for user `2bb84dc148d9498787752dea558edef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:50.915345 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e4c017b3-7302-411e-9997-831055829190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:50.997783 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1000/2008] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:51.002673 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc2e36-5741-4a97-ac2c-bd7d7141e4c1 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:51.073807 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc2e36-5741-4a97-ac2c-bd7d7141e4c1 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:51.078608 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd091be-6bbd-4f2d-a7c5-e46ba7407d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:51.079267 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd091be-6bbd-4f2d-a7c5-e46ba7407d2a admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:51.079553 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd091be-6bbd-4f2d-a7c5-e46ba7407d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:51.079831 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd091be-6bbd-4f2d-a7c5-e46ba7407d2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:51.080538 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd091be-6bbd-4f2d-a7c5-e46ba7407d2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:51.082489 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd091be-6bbd-4f2d-a7c5-e46ba7407d2a admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:51.107238 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1009/2009] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:51 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:38:51.111585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c017b3-7302-411e-9997-831055829190 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:51.122240 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c017b3-7302-411e-9997-831055829190 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:51.134511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:51.182925 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cdb844-8db8-4b71-8af4-88bcb5cc6ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:51.183594 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cdb844-8db8-4b71-8af4-88bcb5cc6ca6 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:51.183875 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cdb844-8db8-4b71-8af4-88bcb5cc6ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:51.184166 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cdb844-8db8-4b71-8af4-88bcb5cc6ca6 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:51.184845 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cdb844-8db8-4b71-8af4-88bcb5cc6ca6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:51.186790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cdb844-8db8-4b71-8af4-88bcb5cc6ca6 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:51.205782 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1001/2010] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:51 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:51.209806 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd091be-6bbd-4f2d-a7c5-e46ba7407d2a admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:51.220883 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd091be-6bbd-4f2d-a7c5-e46ba7407d2a admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:51.225633 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b60ea-3b52-4deb-b575-2c736b9fa70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:51.226321 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b60ea-3b52-4deb-b575-2c736b9fa70f admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:51.226604 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b60ea-3b52-4deb-b575-2c736b9fa70f admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:51.226881 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b60ea-3b52-4deb-b575-2c736b9fa70f admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:51.227609 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4b60ea-3b52-4deb-b575-2c736b9fa70f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:51.229190 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4b60ea-3b52-4deb-b575-2c736b9fa70f admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:51.240623 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1010/2011] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:51.244825 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cdb844-8db8-4b71-8af4-88bcb5cc6ca6 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:51.255053 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cdb844-8db8-4b71-8af4-88bcb5cc6ca6 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:51.259735 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d69db2d-a422-412a-bef7-b8571ef01a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:51.260618 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69db2d-a422-412a-bef7-b8571ef01a4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:51.260894 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69db2d-a422-412a-bef7-b8571ef01a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:51.261162 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69db2d-a422-412a-bef7-b8571ef01a4f admin admin] PATH_INFO: `/v3/projects/fd0a68568cbe40fa9657393a641ddd4d/users/ec763834c48a4304ab2964c3b724b805/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:51.276913 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d69db2d-a422-412a-bef7-b8571ef01a4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd0a68568cbe40fa9657393a641ddd4d, user_id=ec763834c48a4304ab2964c3b724b805, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:51.279634 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d69db2d-a422-412a-bef7-b8571ef01a4f admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:51.293726 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1002/2012] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:51 2022] PUT /identity/v3/projects/fd0a68568cbe40fa9657393a641ddd4d/users/ec763834c48a4304ab2964c3b724b805/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:38:51.297552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b60ea-3b52-4deb-b575-2c736b9fa70f admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:51.308149 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b60ea-3b52-4deb-b575-2c736b9fa70f admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:51.350066 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:51.378885 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa134c64-8645-4327-b52b-9d004b94d129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:51.379569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa134c64-8645-4327-b52b-9d004b94d129 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:51.379856 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa134c64-8645-4327-b52b-9d004b94d129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:51.380136 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa134c64-8645-4327-b52b-9d004b94d129 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:51.380873 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa134c64-8645-4327-b52b-9d004b94d129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:51.382508 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa134c64-8645-4327-b52b-9d004b94d129 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:51.392492 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1011/2013] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:38:51.396465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d69db2d-a422-412a-bef7-b8571ef01a4f admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:51.407059 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d69db2d-a422-412a-bef7-b8571ef01a4f admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:51.411547 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51238770-e324-419d-afc2-b4fd2be0eb7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:51.412413 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51238770-e324-419d-afc2-b4fd2be0eb7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:51.412700 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51238770-e324-419d-afc2-b4fd2be0eb7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:51.412972 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51238770-e324-419d-afc2-b4fd2be0eb7d admin admin] PATH_INFO: `/v3/projects/fd0a68568cbe40fa9657393a641ddd4d/users/ec763834c48a4304ab2964c3b724b805/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:51.414955 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51238770-e324-419d-afc2-b4fd2be0eb7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd0a68568cbe40fa9657393a641ddd4d, user_id=ec763834c48a4304ab2964c3b724b805, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:51.418189 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51238770-e324-419d-afc2-b4fd2be0eb7d admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:51.428141 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1003/2014] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:51 2022] PUT /identity/v3/projects/fd0a68568cbe40fa9657393a641ddd4d/users/ec763834c48a4304ab2964c3b724b805/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:51.438028 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6a8fd-b89b-4da1-904f-f87780df686b None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:51.438354 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6a8fd-b89b-4da1-904f-f87780df686b None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:51.438637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6a8fd-b89b-4da1-904f-f87780df686b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:51.465219 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-91e6a8fd-b89b-4da1-904f-f87780df686b None None] MFA Rules not processed for user `ec763834c48a4304ab2964c3b724b805`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:51.466460 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-91e6a8fd-b89b-4da1-904f-f87780df686b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:51.532569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1012/2015] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:51 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:51.543444 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51238770-e324-419d-afc2-b4fd2be0eb7d admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:51.545122 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-51238770-e324-419d-afc2-b4fd2be0eb7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:51.601450 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51238770-e324-419d-afc2-b4fd2be0eb7d admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:51.605657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1475a38-1ccc-4d65-9c27-e758d10fb935 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:51.606325 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1475a38-1ccc-4d65-9c27-e758d10fb935 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:51.606614 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1475a38-1ccc-4d65-9c27-e758d10fb935 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:51.606895 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1475a38-1ccc-4d65-9c27-e758d10fb935 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:51.615742 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1475a38-1ccc-4d65-9c27-e758d10fb935 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:51.617165 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1475a38-1ccc-4d65-9c27-e758d10fb935 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:51.627901 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1004/2016] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:51 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:38:52.064493 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e6a8fd-b89b-4da1-904f-f87780df686b None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:52.144127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e6a8fd-b89b-4da1-904f-f87780df686b None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:52.149155 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789deef-937d-4dfd-a351-23972e7f0f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:52.150117 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789deef-937d-4dfd-a351-23972e7f0f06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:52.150478 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789deef-937d-4dfd-a351-23972e7f0f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:52.150856 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789deef-937d-4dfd-a351-23972e7f0f06 admin admin] PATH_INFO: `/v3/users/2bb84dc148d9498787752dea558edef0` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:52.152289 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c789deef-937d-4dfd-a351-23972e7f0f06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bb84dc148d9498787752dea558edef0)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:52.154334 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c789deef-937d-4dfd-a351-23972e7f0f06 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:52.238568 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-c789deef-937d-4dfd-a351-23972e7f0f06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bb84dc148d9498787752dea558edef0'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:52.239439 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-c789deef-937d-4dfd-a351-23972e7f0f06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bb84dc148d9498787752dea558edef0'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:52.248356 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-c789deef-937d-4dfd-a351-23972e7f0f06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bb84dc148d9498787752dea558edef0'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:52.249100 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-c789deef-937d-4dfd-a351-23972e7f0f06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bb84dc148d9498787752dea558edef0'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:52.255292 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:52.267138 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-c789deef-937d-4dfd-a351-23972e7f0f06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bb84dc148d9498787752dea558edef0'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:52.289795 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1013/2017] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:38:52 2022] DELETE /identity/v3/users/2bb84dc148d9498787752dea558edef0 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:52.292765 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273cbfb-7888-4141-824a-b60b39f5da8b None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:52.293271 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273cbfb-7888-4141-824a-b60b39f5da8b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:52.293401 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273cbfb-7888-4141-824a-b60b39f5da8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:52.315159 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-e273cbfb-7888-4141-824a-b60b39f5da8b None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:52.316555 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e273cbfb-7888-4141-824a-b60b39f5da8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:52.405018 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1005/2018] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:52.414975 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789deef-937d-4dfd-a351-23972e7f0f06 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:52.417955 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-c789deef-937d-4dfd-a351-23972e7f0f06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:52.499056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789deef-937d-4dfd-a351-23972e7f0f06 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:52.503732 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ea68e-1bd4-42be-82b1-0489f2df9318 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:52.504569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ea68e-1bd4-42be-82b1-0489f2df9318 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:52.504857 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ea68e-1bd4-42be-82b1-0489f2df9318 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:52.505090 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ea68e-1bd4-42be-82b1-0489f2df9318 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:52.519002 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ea68e-1bd4-42be-82b1-0489f2df9318 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:52.521502 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ea68e-1bd4-42be-82b1-0489f2df9318 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:52.535667 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1014/2019] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:38:52.612397 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e273cbfb-7888-4141-824a-b60b39f5da8b None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:52.613942 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e273cbfb-7888-4141-824a-b60b39f5da8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:52.634126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e273cbfb-7888-4141-824a-b60b39f5da8b None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:52.643511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887cf4f-9e6c-4480-a933-ab9df00f65d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:52.645672 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887cf4f-9e6c-4480-a933-ab9df00f65d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:52.645672 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887cf4f-9e6c-4480-a933-ab9df00f65d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:52.646264 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887cf4f-9e6c-4480-a933-ab9df00f65d4 admin admin] PATH_INFO: `/v3/users/c792103be95040719c1772cd8aa1d12f` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:52.648244 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b887cf4f-9e6c-4480-a933-ab9df00f65d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c792103be95040719c1772cd8aa1d12f)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:52.652458 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b887cf4f-9e6c-4480-a933-ab9df00f65d4 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:52.667807 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:52.719863 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-b887cf4f-9e6c-4480-a933-ab9df00f65d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c792103be95040719c1772cd8aa1d12f'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:52.720907 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-b887cf4f-9e6c-4480-a933-ab9df00f65d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c792103be95040719c1772cd8aa1d12f'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:52.721697 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-b887cf4f-9e6c-4480-a933-ab9df00f65d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c792103be95040719c1772cd8aa1d12f'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:52.737616 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-b887cf4f-9e6c-4480-a933-ab9df00f65d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c792103be95040719c1772cd8aa1d12f'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:52.746404 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-b887cf4f-9e6c-4480-a933-ab9df00f65d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c792103be95040719c1772cd8aa1d12f'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:52.760041 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1006/2020] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:38:52 2022] DELETE /identity/v3/users/c792103be95040719c1772cd8aa1d12f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:52.861934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887cf4f-9e6c-4480-a933-ab9df00f65d4 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:52.862606 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ea68e-1bd4-42be-82b1-0489f2df9318 service neutron] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:52.864594 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-679ea68e-1bd4-42be-82b1-0489f2df9318 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:52.866284 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b887cf4f-9e6c-4480-a933-ab9df00f65d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:52.893919 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ea68e-1bd4-42be-82b1-0489f2df9318 service neutron] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:52.959645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb9fdf9-b7be-4442-8f4f-56b3627236f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:52.959957 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887cf4f-9e6c-4480-a933-ab9df00f65d4 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:52.960587 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9fdf9-b7be-4442-8f4f-56b3627236f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:52.960875 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9fdf9-b7be-4442-8f4f-56b3627236f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:52.961155 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9fdf9-b7be-4442-8f4f-56b3627236f8 admin admin] PATH_INFO: `/v3/projects/b7fa1b5b797542f7a65a56df53b0c5d8` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:52.962604 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb9fdf9-b7be-4442-8f4f-56b3627236f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7fa1b5b797542f7a65a56df53b0c5d8)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:52.964693 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb9fdf9-b7be-4442-8f4f-56b3627236f8 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:52.965802 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003214c-dec4-40ac-be0f-f8453ae73902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:52.967091 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003214c-dec4-40ac-be0f-f8453ae73902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:52.967091 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003214c-dec4-40ac-be0f-f8453ae73902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:52.967284 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003214c-dec4-40ac-be0f-f8453ae73902 admin admin] PATH_INFO: `/v3/users/2589331db7ba4ed3904df67e89b9f896` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:52.968511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6003214c-dec4-40ac-be0f-f8453ae73902 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2589331db7ba4ed3904df67e89b9f896)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:52.970487 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6003214c-dec4-40ac-be0f-f8453ae73902 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:53.020784 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-8cb9fdf9-b7be-4442-8f4f-56b3627236f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7fa1b5b797542f7a65a56df53b0c5d8', 'request_id': 'req-8cb9fdf9-b7be-4442-8f4f-56b3627236f8'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:53.028667 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-8cb9fdf9-b7be-4442-8f4f-56b3627236f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7fa1b5b797542f7a65a56df53b0c5d8', 'request_id': 'req-8cb9fdf9-b7be-4442-8f4f-56b3627236f8'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:53.028817 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6003214c-dec4-40ac-be0f-f8453ae73902 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2589331db7ba4ed3904df67e89b9f896'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:53.029446 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6003214c-dec4-40ac-be0f-f8453ae73902 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2589331db7ba4ed3904df67e89b9f896'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:53.030071 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6003214c-dec4-40ac-be0f-f8453ae73902 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2589331db7ba4ed3904df67e89b9f896'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:53.035041 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-8cb9fdf9-b7be-4442-8f4f-56b3627236f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7fa1b5b797542f7a65a56df53b0c5d8', 'request_id': 'req-8cb9fdf9-b7be-4442-8f4f-56b3627236f8'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:53.042959 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6003214c-dec4-40ac-be0f-f8453ae73902 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2589331db7ba4ed3904df67e89b9f896'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:53.046639 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-8cb9fdf9-b7be-4442-8f4f-56b3627236f8 admin admin] The token cache is being invalidate because project b7fa1b5b797542f7a65a56df53b0c5d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:53.046929 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-8cb9fdf9-b7be-4442-8f4f-56b3627236f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:53.047653 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-8cb9fdf9-b7be-4442-8f4f-56b3627236f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:53.049294 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1015/2021] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:52 2022] DELETE /identity/v3/projects/b7fa1b5b797542f7a65a56df53b0c5d8 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:53.051004 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6003214c-dec4-40ac-be0f-f8453ae73902 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2589331db7ba4ed3904df67e89b9f896'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:53.051412 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb9fdf9-b7be-4442-8f4f-56b3627236f8 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:53.052926 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-8cb9fdf9-b7be-4442-8f4f-56b3627236f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:53.064432 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:53.065354 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1007/2022] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:38:52 2022] DELETE /identity/v3/users/2589331db7ba4ed3904df67e89b9f896 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:53.066292 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003214c-dec4-40ac-be0f-f8453ae73902 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:53.067656 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6003214c-dec4-40ac-be0f-f8453ae73902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:53.073934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:53.123896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb9fdf9-b7be-4442-8f4f-56b3627236f8 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:53.129043 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c914c-cddf-477d-9747-c702c29a38c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:53.129908 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c914c-cddf-477d-9747-c702c29a38c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:53.130220 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c914c-cddf-477d-9747-c702c29a38c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:53.130503 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c914c-cddf-477d-9747-c702c29a38c5 admin admin] PATH_INFO: `/v3/projects/240ef47b993b48dc9f48932a3c077a9a` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:53.131692 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2c914c-cddf-477d-9747-c702c29a38c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=240ef47b993b48dc9f48932a3c077a9a)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:53.133587 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2c914c-cddf-477d-9747-c702c29a38c5 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:53.134526 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003214c-dec4-40ac-be0f-f8453ae73902 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:53.140986 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45cf0c8-06d2-4f72-9dd0-06a1258457ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:53.141784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45cf0c8-06d2-4f72-9dd0-06a1258457ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:53.142184 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45cf0c8-06d2-4f72-9dd0-06a1258457ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:53.142467 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45cf0c8-06d2-4f72-9dd0-06a1258457ae admin admin] PATH_INFO: `/v3/users/ec763834c48a4304ab2964c3b724b805` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:53.144295 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45cf0c8-06d2-4f72-9dd0-06a1258457ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec763834c48a4304ab2964c3b724b805)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:53.146405 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45cf0c8-06d2-4f72-9dd0-06a1258457ae admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:53.181823 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-eb2c914c-cddf-477d-9747-c702c29a38c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '240ef47b993b48dc9f48932a3c077a9a', 'request_id': 'req-eb2c914c-cddf-477d-9747-c702c29a38c5'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:53.190214 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-eb2c914c-cddf-477d-9747-c702c29a38c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '240ef47b993b48dc9f48932a3c077a9a', 'request_id': 'req-eb2c914c-cddf-477d-9747-c702c29a38c5'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:53.194934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-c45cf0c8-06d2-4f72-9dd0-06a1258457ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec763834c48a4304ab2964c3b724b805'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:53.195591 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-c45cf0c8-06d2-4f72-9dd0-06a1258457ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec763834c48a4304ab2964c3b724b805'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:53.196215 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-c45cf0c8-06d2-4f72-9dd0-06a1258457ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec763834c48a4304ab2964c3b724b805'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:53.196584 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-eb2c914c-cddf-477d-9747-c702c29a38c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '240ef47b993b48dc9f48932a3c077a9a', 'request_id': 'req-eb2c914c-cddf-477d-9747-c702c29a38c5'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:53.207734 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-eb2c914c-cddf-477d-9747-c702c29a38c5 admin admin] The token cache is being invalidate because project 240ef47b993b48dc9f48932a3c077a9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:53.208018 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-eb2c914c-cddf-477d-9747-c702c29a38c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:53.208667 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-eb2c914c-cddf-477d-9747-c702c29a38c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:53.208777 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-c45cf0c8-06d2-4f72-9dd0-06a1258457ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec763834c48a4304ab2964c3b724b805'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:53.214703 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1016/2023] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:53 2022] DELETE /identity/v3/projects/240ef47b993b48dc9f48932a3c077a9a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:53.214703 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eda43a-a581-4b3c-aed7-e93d005deedd None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:53.214703 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eda43a-a581-4b3c-aed7-e93d005deedd None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:53.214703 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eda43a-a581-4b3c-aed7-e93d005deedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:53.216548 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-c45cf0c8-06d2-4f72-9dd0-06a1258457ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec763834c48a4304ab2964c3b724b805'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:53.228884 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-f4eda43a-a581-4b3c-aed7-e93d005deedd None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:53.230655 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-f4eda43a-a581-4b3c-aed7-e93d005deedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:53.231091 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1008/2024] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:38:53 2022] DELETE /identity/v3/users/ec763834c48a4304ab2964c3b724b805 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:53.232292 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54335572-e4cb-49ab-9e61-4d9de29f9b70 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:53.232586 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54335572-e4cb-49ab-9e61-4d9de29f9b70 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:53.232862 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54335572-e4cb-49ab-9e61-4d9de29f9b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:53.253901 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-54335572-e4cb-49ab-9e61-4d9de29f9b70 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:53.255144 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-54335572-e4cb-49ab-9e61-4d9de29f9b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:53.301926 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1017/2025] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:53.314003 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eda43a-a581-4b3c-aed7-e93d005deedd None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:53.316253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-f4eda43a-a581-4b3c-aed7-e93d005deedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:53.328189 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1009/2026] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:53.336358 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b470d70-6c3b-47e4-bd8d-3dc87e6458b7 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:53.336652 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b470d70-6c3b-47e4-bd8d-3dc87e6458b7 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:53.336929 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b470d70-6c3b-47e4-bd8d-3dc87e6458b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:53.356407 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-8b470d70-6c3b-47e4-bd8d-3dc87e6458b7 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:53.357629 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8b470d70-6c3b-47e4-bd8d-3dc87e6458b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:53.366132 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1010/2027] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:53.370661 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b470d70-6c3b-47e4-bd8d-3dc87e6458b7 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:53.379897 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b470d70-6c3b-47e4-bd8d-3dc87e6458b7 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:53.387779 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a208606f-354b-4944-8c42-4d44725885ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:53.387779 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208606f-354b-4944-8c42-4d44725885ed admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:53.387779 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208606f-354b-4944-8c42-4d44725885ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:53.387779 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208606f-354b-4944-8c42-4d44725885ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:53.388751 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a208606f-354b-4944-8c42-4d44725885ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:38:53.388751 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a208606f-354b-4944-8c42-4d44725885ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:53.388751 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a208606f-354b-4944-8c42-4d44725885ed admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:53.390201 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:53.401880 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1011/2028] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:38:53.406233 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a208606f-354b-4944-8c42-4d44725885ed admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:53.414761 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:53.419342 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a208606f-354b-4944-8c42-4d44725885ed admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:53.419693 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eda43a-a581-4b3c-aed7-e93d005deedd None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:53.423771 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99442e4e-12e0-4748-bde3-6acaf194b013 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:53.424432 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99442e4e-12e0-4748-bde3-6acaf194b013 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:53.424713 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99442e4e-12e0-4748-bde3-6acaf194b013 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:53.424991 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99442e4e-12e0-4748-bde3-6acaf194b013 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:53.427800 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f8aa8-0eba-4f64-af57-46289719d328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:53.428761 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f8aa8-0eba-4f64-af57-46289719d328 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:53.429048 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f8aa8-0eba-4f64-af57-46289719d328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:53.429327 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f8aa8-0eba-4f64-af57-46289719d328 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:53.431543 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f8aa8-0eba-4f64-af57-46289719d328 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:53.439041 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99442e4e-12e0-4748-bde3-6acaf194b013 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:53.440632 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99442e4e-12e0-4748-bde3-6acaf194b013 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:53.441140 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f8aa8-0eba-4f64-af57-46289719d328 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:53.462177 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1018/2029] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:38:53.475529 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1012/2030] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:53 2022] POST /identity/v3/projects => generated 381 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:38:53.479973 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99442e4e-12e0-4748-bde3-6acaf194b013 service neutron] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:53.492261 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99442e4e-12e0-4748-bde3-6acaf194b013 service neutron] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:53.546399 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f8aa8-0eba-4f64-af57-46289719d328 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:53.548241 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-361f8aa8-0eba-4f64-af57-46289719d328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:53.577389 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4877a902-79cc-44fa-b3b3-49978b9f79d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:53.578164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877a902-79cc-44fa-b3b3-49978b9f79d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:53.578743 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877a902-79cc-44fa-b3b3-49978b9f79d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:53.578743 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877a902-79cc-44fa-b3b3-49978b9f79d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:53.579433 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4877a902-79cc-44fa-b3b3-49978b9f79d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:53.581505 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4877a902-79cc-44fa-b3b3-49978b9f79d1 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:53.613085 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1019/2031] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:53 2022] POST /identity/v3/users => generated 291 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:38:53.626336 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4877a902-79cc-44fa-b3b3-49978b9f79d1 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:53.636732 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4877a902-79cc-44fa-b3b3-49978b9f79d1 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:53.643649 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa55e08-8753-48b9-8eb2-80bca8a031ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:53.644434 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa55e08-8753-48b9-8eb2-80bca8a031ad admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:53.644877 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa55e08-8753-48b9-8eb2-80bca8a031ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:53.645204 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa55e08-8753-48b9-8eb2-80bca8a031ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:53.646841 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa55e08-8753-48b9-8eb2-80bca8a031ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:53.651920 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa55e08-8753-48b9-8eb2-80bca8a031ad admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:53.666515 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1020/2032] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:53.671625 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa55e08-8753-48b9-8eb2-80bca8a031ad admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:53.687383 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa55e08-8753-48b9-8eb2-80bca8a031ad admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:53.695799 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dddad5-9338-408c-9971-42fe0473bcae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:53.696706 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dddad5-9338-408c-9971-42fe0473bcae admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:53.697105 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dddad5-9338-408c-9971-42fe0473bcae admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:53.697460 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dddad5-9338-408c-9971-42fe0473bcae admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:53.698850 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dddad5-9338-408c-9971-42fe0473bcae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:53.700532 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dddad5-9338-408c-9971-42fe0473bcae admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:53.715206 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f8aa8-0eba-4f64-af57-46289719d328 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:53.718055 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1021/2033] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:53.719173 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dddad5-9338-408c-9971-42fe0473bcae admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:53.721066 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a4cbb-f2de-4246-a672-57185c11a585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:53.721214 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-41dddad5-9338-408c-9971-42fe0473bcae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:53.722192 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a4cbb-f2de-4246-a672-57185c11a585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:53.722557 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a4cbb-f2de-4246-a672-57185c11a585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:53.722979 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a4cbb-f2de-4246-a672-57185c11a585 admin admin] PATH_INFO: `/v3/projects/fd0a68568cbe40fa9657393a641ddd4d` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:53.724480 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a4cbb-f2de-4246-a672-57185c11a585 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd0a68568cbe40fa9657393a641ddd4d)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:53.727190 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a4cbb-f2de-4246-a672-57185c11a585 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:53.735796 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dddad5-9338-408c-9971-42fe0473bcae admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:53.742477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a95f9-2b4f-4ce6-b894-a821608c6cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:53.746495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a95f9-2b4f-4ce6-b894-a821608c6cb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:53.746779 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a95f9-2b4f-4ce6-b894-a821608c6cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:53.747053 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a95f9-2b4f-4ce6-b894-a821608c6cb8 admin admin] PATH_INFO: `/v3/users/d108703b84904b04b52ac17892b30a75` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:53.748420 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a95f9-2b4f-4ce6-b894-a821608c6cb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d108703b84904b04b52ac17892b30a75)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:53.750644 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a95f9-2b4f-4ce6-b894-a821608c6cb8 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:53.802419 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6e4a4cbb-f2de-4246-a672-57185c11a585 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd0a68568cbe40fa9657393a641ddd4d', 'request_id': 'req-6e4a4cbb-f2de-4246-a672-57185c11a585'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:53.812368 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6e4a4cbb-f2de-4246-a672-57185c11a585 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd0a68568cbe40fa9657393a641ddd4d', 'request_id': 'req-6e4a4cbb-f2de-4246-a672-57185c11a585'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:53.815867 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-002a95f9-2b4f-4ce6-b894-a821608c6cb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd108703b84904b04b52ac17892b30a75'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:53.818795 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-002a95f9-2b4f-4ce6-b894-a821608c6cb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd108703b84904b04b52ac17892b30a75'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:53.819685 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6e4a4cbb-f2de-4246-a672-57185c11a585 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd0a68568cbe40fa9657393a641ddd4d', 'request_id': 'req-6e4a4cbb-f2de-4246-a672-57185c11a585'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:53.827237 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-002a95f9-2b4f-4ce6-b894-a821608c6cb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd108703b84904b04b52ac17892b30a75'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:53.827905 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-002a95f9-2b4f-4ce6-b894-a821608c6cb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd108703b84904b04b52ac17892b30a75'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:53.833723 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6e4a4cbb-f2de-4246-a672-57185c11a585 admin admin] The token cache is being invalidate because project fd0a68568cbe40fa9657393a641ddd4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:53.834398 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6e4a4cbb-f2de-4246-a672-57185c11a585 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:53.835298 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6e4a4cbb-f2de-4246-a672-57185c11a585 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:53.836838 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-002a95f9-2b4f-4ce6-b894-a821608c6cb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd108703b84904b04b52ac17892b30a75'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:53.837833 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1013/2034] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:53 2022] DELETE /identity/v3/projects/fd0a68568cbe40fa9657393a641ddd4d => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:38:53.838599 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a4cbb-f2de-4246-a672-57185c11a585 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:53.840589 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6e4a4cbb-f2de-4246-a672-57185c11a585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:53.859215 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1022/2035] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:38:53 2022] DELETE /identity/v3/users/d108703b84904b04b52ac17892b30a75 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:53.864117 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13dffcc-6e2f-4fe5-ac8c-d9276b060b66 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:53.864289 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13dffcc-6e2f-4fe5-ac8c-d9276b060b66 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:53.865433 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13dffcc-6e2f-4fe5-ac8c-d9276b060b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:53.902776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-c13dffcc-6e2f-4fe5-ac8c-d9276b060b66 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:53.904053 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-c13dffcc-6e2f-4fe5-ac8c-d9276b060b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:53.924382 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a4cbb-f2de-4246-a672-57185c11a585 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:53.931112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:53.942472 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:53.977828 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1023/2036] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:53.978712 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13dffcc-6e2f-4fe5-ac8c-d9276b060b66 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:53.979974 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-c13dffcc-6e2f-4fe5-ac8c-d9276b060b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:53.986579 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbd5fd9-e1ca-4bde-9322-11bc7379fba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:53.987549 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd5fd9-e1ca-4bde-9322-11bc7379fba4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:53.987828 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd5fd9-e1ca-4bde-9322-11bc7379fba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:53.988126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd5fd9-e1ca-4bde-9322-11bc7379fba4 admin admin] PATH_INFO: `/v3/projects/45247c5e973e47b2be087d24a818400b/users/14970397a0ec49c98c4e82d994a56858/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:53.991312 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13dffcc-6e2f-4fe5-ac8c-d9276b060b66 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:53.995912 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e430701d-99c1-4624-8422-a4afb1d1deb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:53.996810 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430701d-99c1-4624-8422-a4afb1d1deb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:53.997096 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430701d-99c1-4624-8422-a4afb1d1deb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:53.997368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430701d-99c1-4624-8422-a4afb1d1deb2 admin admin] PATH_INFO: `/v3/projects/574db4fd8c7c4742918c3628a3cfa993` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:53.998566 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e430701d-99c1-4624-8422-a4afb1d1deb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=574db4fd8c7c4742918c3628a3cfa993)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:54.000731 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e430701d-99c1-4624-8422-a4afb1d1deb2 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:54.002769 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbd5fd9-e1ca-4bde-9322-11bc7379fba4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45247c5e973e47b2be087d24a818400b, user_id=14970397a0ec49c98c4e82d994a56858, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:54.005620 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbd5fd9-e1ca-4bde-9322-11bc7379fba4 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:54.019878 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1014/2037] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:53 2022] PUT /identity/v3/projects/45247c5e973e47b2be087d24a818400b/users/14970397a0ec49c98c4e82d994a56858/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:54.029147 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680cc6c4-71a9-4e89-805d-d3f217aeefbd None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:54.029147 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680cc6c4-71a9-4e89-805d-d3f217aeefbd None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:54.029147 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680cc6c4-71a9-4e89-805d-d3f217aeefbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:54.044384 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-680cc6c4-71a9-4e89-805d-d3f217aeefbd None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:54.045587 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-680cc6c4-71a9-4e89-805d-d3f217aeefbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:54.051370 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-e430701d-99c1-4624-8422-a4afb1d1deb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '574db4fd8c7c4742918c3628a3cfa993', 'request_id': 'req-e430701d-99c1-4624-8422-a4afb1d1deb2'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:54.059746 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-e430701d-99c1-4624-8422-a4afb1d1deb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '574db4fd8c7c4742918c3628a3cfa993', 'request_id': 'req-e430701d-99c1-4624-8422-a4afb1d1deb2'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:54.066004 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-e430701d-99c1-4624-8422-a4afb1d1deb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '574db4fd8c7c4742918c3628a3cfa993', 'request_id': 'req-e430701d-99c1-4624-8422-a4afb1d1deb2'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:54.076805 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-e430701d-99c1-4624-8422-a4afb1d1deb2 admin admin] The token cache is being invalidate because project 574db4fd8c7c4742918c3628a3cfa993 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:54.077077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-e430701d-99c1-4624-8422-a4afb1d1deb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:54.077728 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-e430701d-99c1-4624-8422-a4afb1d1deb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:54.079329 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1024/2038] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:53 2022] DELETE /identity/v3/projects/574db4fd8c7c4742918c3628a3cfa993 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:54.080547 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe3a62-f3e1-4b46-9c07-002b97d092a0 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:54.080829 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe3a62-f3e1-4b46-9c07-002b97d092a0 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:54.081101 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe3a62-f3e1-4b46-9c07-002b97d092a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:54.099620 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-8ebe3a62-f3e1-4b46-9c07-002b97d092a0 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:54.101042 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-8ebe3a62-f3e1-4b46-9c07-002b97d092a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:54.107793 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1015/2039] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:54.109128 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccdb59b-3d01-4096-822e-2ed2bb2626e1 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:54.109416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccdb59b-3d01-4096-822e-2ed2bb2626e1 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:54.109687 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccdb59b-3d01-4096-822e-2ed2bb2626e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:54.112220 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1025/2040] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:54.112943 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe3a62-f3e1-4b46-9c07-002b97d092a0 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:54.114203 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-8ebe3a62-f3e1-4b46-9c07-002b97d092a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:54.126692 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe3a62-f3e1-4b46-9c07-002b97d092a0 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:54.132069 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196752bd-8df4-4263-befb-eb814bd27b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:54.132928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196752bd-8df4-4263-befb-eb814bd27b05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:54.133205 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196752bd-8df4-4263-befb-eb814bd27b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:54.133482 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196752bd-8df4-4263-befb-eb814bd27b05 admin admin] PATH_INFO: `/v3/users/416fb78b591041e09ef42b08534bbf04` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:54.134704 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196752bd-8df4-4263-befb-eb814bd27b05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=416fb78b591041e09ef42b08534bbf04)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:54.137207 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196752bd-8df4-4263-befb-eb814bd27b05 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:54.137429 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-fccdb59b-3d01-4096-822e-2ed2bb2626e1 None None] MFA Rules not processed for user `14970397a0ec49c98c4e82d994a56858`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:54.138646 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-fccdb59b-3d01-4096-822e-2ed2bb2626e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:54.186119 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-196752bd-8df4-4263-befb-eb814bd27b05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '416fb78b591041e09ef42b08534bbf04'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:54.186701 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-196752bd-8df4-4263-befb-eb814bd27b05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '416fb78b591041e09ef42b08534bbf04'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:54.194179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-196752bd-8df4-4263-befb-eb814bd27b05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '416fb78b591041e09ef42b08534bbf04'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:54.194878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-196752bd-8df4-4263-befb-eb814bd27b05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '416fb78b591041e09ef42b08534bbf04'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:54.202555 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-196752bd-8df4-4263-befb-eb814bd27b05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '416fb78b591041e09ef42b08534bbf04'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:54.205750 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1016/2041] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:54.209698 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccdb59b-3d01-4096-822e-2ed2bb2626e1 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:54.209698 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-fccdb59b-3d01-4096-822e-2ed2bb2626e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:54.220601 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccdb59b-3d01-4096-822e-2ed2bb2626e1 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:54.223081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1026/2042] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:38:54 2022] DELETE /identity/v3/users/416fb78b591041e09ef42b08534bbf04 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:54.223890 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196752bd-8df4-4263-befb-eb814bd27b05 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:54.227017 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-196752bd-8df4-4263-befb-eb814bd27b05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:54.228095 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fdb818-2e40-4371-9308-8034f5cdba7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:54.228729 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fdb818-2e40-4371-9308-8034f5cdba7a admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:54.229012 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fdb818-2e40-4371-9308-8034f5cdba7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:54.229309 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fdb818-2e40-4371-9308-8034f5cdba7a admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:54.230041 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fdb818-2e40-4371-9308-8034f5cdba7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:38:54.230392 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fdb818-2e40-4371-9308-8034f5cdba7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:54.232003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fdb818-2e40-4371-9308-8034f5cdba7a admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:54.242368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:54.244508 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1017/2043] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:38:54.246121 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fdb818-2e40-4371-9308-8034f5cdba7a admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:54.249971 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-10fdb818-2e40-4371-9308-8034f5cdba7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:54.265106 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fdb818-2e40-4371-9308-8034f5cdba7a admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:54.284477 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:54.304923 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196752bd-8df4-4263-befb-eb814bd27b05 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:54.309478 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac2c6f-6af2-4d54-aced-99b50db075b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:54.310008 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac2c6f-6af2-4d54-aced-99b50db075b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:54.310930 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac2c6f-6af2-4d54-aced-99b50db075b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:54.311440 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac2c6f-6af2-4d54-aced-99b50db075b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:54.313755 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-35ac2c6f-6af2-4d54-aced-99b50db075b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:54.332224 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e74f34-7547-46ff-a936-1cbfdb3a32ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:54.333935 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e74f34-7547-46ff-a936-1cbfdb3a32ef admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:54.334416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e74f34-7547-46ff-a936-1cbfdb3a32ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:54.334739 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e74f34-7547-46ff-a936-1cbfdb3a32ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:54.342192 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e74f34-7547-46ff-a936-1cbfdb3a32ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:54.342192 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e74f34-7547-46ff-a936-1cbfdb3a32ef admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:54.375558 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1018/2044] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:54 2022] POST /identity/v3/projects => generated 383 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:54.376863 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e74f34-7547-46ff-a936-1cbfdb3a32ef admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:54.406518 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ac2c6f-6af2-4d54-aced-99b50db075b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:54.408422 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ac2c6f-6af2-4d54-aced-99b50db075b7 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:54.420498 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1027/2045] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:38:54.421517 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac2c6f-6af2-4d54-aced-99b50db075b7 service neutron] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:54.432971 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac2c6f-6af2-4d54-aced-99b50db075b7 service neutron] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:54.438210 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eeac51-9051-4bf4-99b8-36f7045fae5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:54.439495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eeac51-9051-4bf4-99b8-36f7045fae5a admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:54.439823 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eeac51-9051-4bf4-99b8-36f7045fae5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:54.440144 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eeac51-9051-4bf4-99b8-36f7045fae5a admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:54.441183 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eeac51-9051-4bf4-99b8-36f7045fae5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:54.443313 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e74f34-7547-46ff-a936-1cbfdb3a32ef admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:54.443428 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eeac51-9051-4bf4-99b8-36f7045fae5a admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:54.449358 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553bae1-41b4-45e9-b3a9-2bc1b697a739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:54.450297 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553bae1-41b4-45e9-b3a9-2bc1b697a739 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:54.450851 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553bae1-41b4-45e9-b3a9-2bc1b697a739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:54.451292 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553bae1-41b4-45e9-b3a9-2bc1b697a739 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:54.452137 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0553bae1-41b4-45e9-b3a9-2bc1b697a739 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:54.454548 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0553bae1-41b4-45e9-b3a9-2bc1b697a739 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:54.476305 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1028/2046] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:54 2022] POST /identity/v3/users => generated 298 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:54.480299 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eeac51-9051-4bf4-99b8-36f7045fae5a admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:54.493025 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eeac51-9051-4bf4-99b8-36f7045fae5a admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:54.499030 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1019/2047] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:54 2022] POST /identity/v3/projects => generated 377 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:54.503146 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200c96dd-bd3f-444d-9ac4-50d7892775ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:54.503146 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c96dd-bd3f-444d-9ac4-50d7892775ef admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:54.503146 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c96dd-bd3f-444d-9ac4-50d7892775ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:54.503146 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c96dd-bd3f-444d-9ac4-50d7892775ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:54.503481 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200c96dd-bd3f-444d-9ac4-50d7892775ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:54.504061 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200c96dd-bd3f-444d-9ac4-50d7892775ef admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:54.504777 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553bae1-41b4-45e9-b3a9-2bc1b697a739 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:54.519930 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1029/2048] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:54.521530 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200c96dd-bd3f-444d-9ac4-50d7892775ef admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:54.534539 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200c96dd-bd3f-444d-9ac4-50d7892775ef admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:54.601071 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553bae1-41b4-45e9-b3a9-2bc1b697a739 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:54.607341 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a0cc36-c39c-4989-b66f-f483d1465527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:54.608221 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0cc36-c39c-4989-b66f-f483d1465527 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:54.608578 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0cc36-c39c-4989-b66f-f483d1465527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:54.608925 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0cc36-c39c-4989-b66f-f483d1465527 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:54.609818 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a0cc36-c39c-4989-b66f-f483d1465527 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:54.611968 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a0cc36-c39c-4989-b66f-f483d1465527 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:54.640491 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95efe3f9-66d7-46a6-a09a-f610a66f84ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:54.641554 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95efe3f9-66d7-46a6-a09a-f610a66f84ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:54.641888 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1020/2049] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:54 2022] POST /identity/v3/users => generated 289 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:54.642008 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95efe3f9-66d7-46a6-a09a-f610a66f84ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:54.642470 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95efe3f9-66d7-46a6-a09a-f610a66f84ba admin admin] PATH_INFO: `/v3/projects/592f1a334ef34db6ad1fdea5154fcc26/users/0faeee214e1544bfba1e9c8c12cf05d0/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:54.642691 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a0cc36-c39c-4989-b66f-f483d1465527 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:54.644380 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-90a0cc36-c39c-4989-b66f-f483d1465527 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:54.661206 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a0cc36-c39c-4989-b66f-f483d1465527 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:54.663005 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95efe3f9-66d7-46a6-a09a-f610a66f84ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=592f1a334ef34db6ad1fdea5154fcc26, user_id=0faeee214e1544bfba1e9c8c12cf05d0, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:54.666299 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fea6062-67e1-46d5-8d6f-cedb988bc92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:54.666435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95efe3f9-66d7-46a6-a09a-f610a66f84ba admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:54.667875 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea6062-67e1-46d5-8d6f-cedb988bc92b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:54.668224 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea6062-67e1-46d5-8d6f-cedb988bc92b admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:54.668599 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea6062-67e1-46d5-8d6f-cedb988bc92b admin admin] PATH_INFO: `/v3/projects/a77989ad54e24c4097fbfc6b6af445b9` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:54.669893 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fea6062-67e1-46d5-8d6f-cedb988bc92b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a77989ad54e24c4097fbfc6b6af445b9)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:54.671929 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fea6062-67e1-46d5-8d6f-cedb988bc92b admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:54.683591 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1030/2050] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:54 2022] PUT /identity/v3/projects/592f1a334ef34db6ad1fdea5154fcc26/users/0faeee214e1544bfba1e9c8c12cf05d0/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:54.686697 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95efe3f9-66d7-46a6-a09a-f610a66f84ba admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:54.693485 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:54.703597 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:54.732424 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8fea6062-67e1-46d5-8d6f-cedb988bc92b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a77989ad54e24c4097fbfc6b6af445b9', 'request_id': 'req-8fea6062-67e1-46d5-8d6f-cedb988bc92b'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:54.743235 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8fea6062-67e1-46d5-8d6f-cedb988bc92b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a77989ad54e24c4097fbfc6b6af445b9', 'request_id': 'req-8fea6062-67e1-46d5-8d6f-cedb988bc92b'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:54.750892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8fea6062-67e1-46d5-8d6f-cedb988bc92b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a77989ad54e24c4097fbfc6b6af445b9', 'request_id': 'req-8fea6062-67e1-46d5-8d6f-cedb988bc92b'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:54.758493 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95efe3f9-66d7-46a6-a09a-f610a66f84ba admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:54.764815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8fea6062-67e1-46d5-8d6f-cedb988bc92b admin admin] The token cache is being invalidate because project a77989ad54e24c4097fbfc6b6af445b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:54.765278 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8fea6062-67e1-46d5-8d6f-cedb988bc92b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:54.766175 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8fea6062-67e1-46d5-8d6f-cedb988bc92b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:54.768512 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1021/2051] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:54 2022] DELETE /identity/v3/projects/a77989ad54e24c4097fbfc6b6af445b9 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:54.772888 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fea6062-67e1-46d5-8d6f-cedb988bc92b admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:54.774129 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8fea6062-67e1-46d5-8d6f-cedb988bc92b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:54.817373 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3c916-c064-418e-aa1e-0cf30f4f79f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:54.818050 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3c916-c064-418e-aa1e-0cf30f4f79f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:54.818366 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3c916-c064-418e-aa1e-0cf30f4f79f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:54.818644 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3c916-c064-418e-aa1e-0cf30f4f79f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:54.819407 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f3c916-c064-418e-aa1e-0cf30f4f79f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:54.820992 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f3c916-c064-418e-aa1e-0cf30f4f79f7 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:54.831388 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1031/2052] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:54.832390 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3c916-c064-418e-aa1e-0cf30f4f79f7 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:54.835883 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-c7f3c916-c064-418e-aa1e-0cf30f4f79f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:54.840399 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fea6062-67e1-46d5-8d6f-cedb988bc92b admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:54.845131 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a990d-93b9-4002-bb31-a38ff8f5c996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:54.845902 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a990d-93b9-4002-bb31-a38ff8f5c996 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:54.846235 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a990d-93b9-4002-bb31-a38ff8f5c996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:54.846508 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a990d-93b9-4002-bb31-a38ff8f5c996 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:54.847297 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353a990d-93b9-4002-bb31-a38ff8f5c996 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:54.848403 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3c916-c064-418e-aa1e-0cf30f4f79f7 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:54.848908 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353a990d-93b9-4002-bb31-a38ff8f5c996 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:54.853127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90babee-36df-487f-aae4-3aa971885d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:54.853965 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90babee-36df-487f-aae4-3aa971885d41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:54.854389 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90babee-36df-487f-aae4-3aa971885d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:54.855177 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90babee-36df-487f-aae4-3aa971885d41 admin admin] PATH_INFO: `/v3/projects/3480b0308f0f44fa84a0ae21f966d71a` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:54.856418 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90babee-36df-487f-aae4-3aa971885d41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3480b0308f0f44fa84a0ae21f966d71a)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:54.858435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90babee-36df-487f-aae4-3aa971885d41 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:54.859482 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1022/2053] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:54.860682 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec8a0a-a7c1-4028-8527-d56c2a22c533 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:54.860972 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec8a0a-a7c1-4028-8527-d56c2a22c533 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:54.861246 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec8a0a-a7c1-4028-8527-d56c2a22c533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:54.879101 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-16ec8a0a-a7c1-4028-8527-d56c2a22c533 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:54.880293 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-16ec8a0a-a7c1-4028-8527-d56c2a22c533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:54.888341 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1023/2054] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:54.890992 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec8a0a-a7c1-4028-8527-d56c2a22c533 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:54.892593 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-16ec8a0a-a7c1-4028-8527-d56c2a22c533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:54.904968 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec8a0a-a7c1-4028-8527-d56c2a22c533 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:54.910124 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-f90babee-36df-487f-aae4-3aa971885d41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3480b0308f0f44fa84a0ae21f966d71a', 'request_id': 'req-f90babee-36df-487f-aae4-3aa971885d41'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:54.919300 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-f90babee-36df-487f-aae4-3aa971885d41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3480b0308f0f44fa84a0ae21f966d71a', 'request_id': 'req-f90babee-36df-487f-aae4-3aa971885d41'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:54.928239 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-f90babee-36df-487f-aae4-3aa971885d41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3480b0308f0f44fa84a0ae21f966d71a', 'request_id': 'req-f90babee-36df-487f-aae4-3aa971885d41'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:54.941496 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-f90babee-36df-487f-aae4-3aa971885d41 admin admin] The token cache is being invalidate because project 3480b0308f0f44fa84a0ae21f966d71a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:54.941891 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-f90babee-36df-487f-aae4-3aa971885d41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:54.942527 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-f90babee-36df-487f-aae4-3aa971885d41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:54.944138 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1032/2055] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:54 2022] DELETE /identity/v3/projects/3480b0308f0f44fa84a0ae21f966d71a => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:54.944820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90babee-36df-487f-aae4-3aa971885d41 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:54.946216 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-f90babee-36df-487f-aae4-3aa971885d41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:54.971611 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb34905b-10e0-4d4e-9640-7a84d267839e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:54.972318 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34905b-10e0-4d4e-9640-7a84d267839e admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:54.972588 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34905b-10e0-4d4e-9640-7a84d267839e admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:54.972911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34905b-10e0-4d4e-9640-7a84d267839e admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:54.973745 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb34905b-10e0-4d4e-9640-7a84d267839e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:54.975298 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb34905b-10e0-4d4e-9640-7a84d267839e admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:54.986748 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1024/2056] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:54.988104 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aecb17-71af-44c6-94ba-6f0b8de730d0 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:54.988399 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aecb17-71af-44c6-94ba-6f0b8de730d0 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:54.988676 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aecb17-71af-44c6-94ba-6f0b8de730d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:54.997880 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:55.008866 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-40aecb17-71af-44c6-94ba-6f0b8de730d0 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:55.010250 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-40aecb17-71af-44c6-94ba-6f0b8de730d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:55.019885 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1025/2057] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:55.020826 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aecb17-71af-44c6-94ba-6f0b8de730d0 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:55.022223 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-40aecb17-71af-44c6-94ba-6f0b8de730d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:55.030227 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90babee-36df-487f-aae4-3aa971885d41 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:55.034980 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aecb17-71af-44c6-94ba-6f0b8de730d0 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:55.036004 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9f975-fc4b-4455-a112-90cc1bb76130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:55.037036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9f975-fc4b-4455-a112-90cc1bb76130 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:55.037387 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9f975-fc4b-4455-a112-90cc1bb76130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:55.037740 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9f975-fc4b-4455-a112-90cc1bb76130 admin admin] PATH_INFO: `/v3/projects/592f1a334ef34db6ad1fdea5154fcc26/users/0faeee214e1544bfba1e9c8c12cf05d0/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:55.039828 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca9f975-fc4b-4455-a112-90cc1bb76130 admin admin] RBAC: Authorizing `identity:create_grant(project_id=592f1a334ef34db6ad1fdea5154fcc26, user_id=0faeee214e1544bfba1e9c8c12cf05d0, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:55.040898 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4209e36f-9e80-4d6e-b1fe-5664cd7927f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:55.042960 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209e36f-9e80-4d6e-b1fe-5664cd7927f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:55.043437 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209e36f-9e80-4d6e-b1fe-5664cd7927f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:55.043862 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209e36f-9e80-4d6e-b1fe-5664cd7927f7 admin admin] PATH_INFO: `/v3/users/e5bf6924564746dc98c77adf6c5904f0` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:55.044034 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca9f975-fc4b-4455-a112-90cc1bb76130 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:55.045480 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4209e36f-9e80-4d6e-b1fe-5664cd7927f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5bf6924564746dc98c77adf6c5904f0)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:55.047627 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4209e36f-9e80-4d6e-b1fe-5664cd7927f7 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:55.056734 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1033/2058] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:54 2022] PUT /identity/v3/projects/592f1a334ef34db6ad1fdea5154fcc26/users/0faeee214e1544bfba1e9c8c12cf05d0/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:55.057698 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9f975-fc4b-4455-a112-90cc1bb76130 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:55.067142 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:55.067872 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-aca9f975-fc4b-4455-a112-90cc1bb76130 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:55.108567 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4209e36f-9e80-4d6e-b1fe-5664cd7927f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5bf6924564746dc98c77adf6c5904f0'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:55.109269 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4209e36f-9e80-4d6e-b1fe-5664cd7927f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5bf6924564746dc98c77adf6c5904f0'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:55.109892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4209e36f-9e80-4d6e-b1fe-5664cd7927f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5bf6924564746dc98c77adf6c5904f0'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:55.122293 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4209e36f-9e80-4d6e-b1fe-5664cd7927f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5bf6924564746dc98c77adf6c5904f0'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:55.129926 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4209e36f-9e80-4d6e-b1fe-5664cd7927f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5bf6924564746dc98c77adf6c5904f0'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:55.135443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9f975-fc4b-4455-a112-90cc1bb76130 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:55.140137 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4d6fca-8b82-48bf-bc26-58c1fdbc3382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:55.141021 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d6fca-8b82-48bf-bc26-58c1fdbc3382 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:55.141303 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d6fca-8b82-48bf-bc26-58c1fdbc3382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:55.141578 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d6fca-8b82-48bf-bc26-58c1fdbc3382 admin admin] PATH_INFO: `/v3/projects/f5a88f690c8b425cac8e649702e157e9/users/c721d29be82a42b0b54ab7d538857649/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:55.143088 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1026/2059] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:38:55 2022] DELETE /identity/v3/users/e5bf6924564746dc98c77adf6c5904f0 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:55.144195 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4209e36f-9e80-4d6e-b1fe-5664cd7927f7 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:55.146509 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-4209e36f-9e80-4d6e-b1fe-5664cd7927f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:55.160844 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4d6fca-8b82-48bf-bc26-58c1fdbc3382 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5a88f690c8b425cac8e649702e157e9, user_id=c721d29be82a42b0b54ab7d538857649, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:55.165317 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4d6fca-8b82-48bf-bc26-58c1fdbc3382 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:55.181870 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1034/2060] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:55 2022] PUT /identity/v3/projects/f5a88f690c8b425cac8e649702e157e9/users/c721d29be82a42b0b54ab7d538857649/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:55.183365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455621b3-9bd0-4fc0-b34a-f4fc11d45298 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:55.183652 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455621b3-9bd0-4fc0-b34a-f4fc11d45298 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:55.183926 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455621b3-9bd0-4fc0-b34a-f4fc11d45298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:55.212092 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-455621b3-9bd0-4fc0-b34a-f4fc11d45298 None None] MFA Rules not processed for user `0faeee214e1544bfba1e9c8c12cf05d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:55.214946 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-455621b3-9bd0-4fc0-b34a-f4fc11d45298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:55.231374 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4209e36f-9e80-4d6e-b1fe-5664cd7927f7 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:55.293304 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6702a8a5-ef38-4962-ae99-e97ff48b8e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:55.294135 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702a8a5-ef38-4962-ae99-e97ff48b8e43 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:55.294432 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702a8a5-ef38-4962-ae99-e97ff48b8e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:55.294709 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702a8a5-ef38-4962-ae99-e97ff48b8e43 admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:55.295445 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6702a8a5-ef38-4962-ae99-e97ff48b8e43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:38:55.295759 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6702a8a5-ef38-4962-ae99-e97ff48b8e43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:55.297376 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6702a8a5-ef38-4962-ae99-e97ff48b8e43 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:55.303204 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1035/2061] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:55 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:55.311192 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1027/2062] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:38:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:38:55.315306 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6702a8a5-ef38-4962-ae99-e97ff48b8e43 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:55.318832 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327ba96-93c1-40bb-a75b-10a1bf266c63 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:55.319125 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327ba96-93c1-40bb-a75b-10a1bf266c63 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:55.319469 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327ba96-93c1-40bb-a75b-10a1bf266c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:55.333628 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6702a8a5-ef38-4962-ae99-e97ff48b8e43 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:55.340032 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55936b3f-4c0e-463e-91d2-1859d73ae8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:55.340706 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55936b3f-4c0e-463e-91d2-1859d73ae8f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:55.340986 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55936b3f-4c0e-463e-91d2-1859d73ae8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:55.341271 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55936b3f-4c0e-463e-91d2-1859d73ae8f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:55.341981 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55936b3f-4c0e-463e-91d2-1859d73ae8f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:55.346073 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55936b3f-4c0e-463e-91d2-1859d73ae8f4 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:55.349801 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-8327ba96-93c1-40bb-a75b-10a1bf266c63 None None] MFA Rules not processed for user `c721d29be82a42b0b54ab7d538857649`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:55.351298 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-8327ba96-93c1-40bb-a75b-10a1bf266c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:55.377629 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1028/2063] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:55 2022] POST /identity/v3/projects => generated 371 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:38:55.378626 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55936b3f-4c0e-463e-91d2-1859d73ae8f4 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:55.380808 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-55936b3f-4c0e-463e-91d2-1859d73ae8f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:55.392520 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:55.397120 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:55.439044 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1036/2064] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:55.440369 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8327ba96-93c1-40bb-a75b-10a1bf266c63 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:55.455898 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8327ba96-93c1-40bb-a75b-10a1bf266c63 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:55.459885 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55936b3f-4c0e-463e-91d2-1859d73ae8f4 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:55.464128 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85980945-05c1-455e-a702-9accf0ca790a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:55.464724 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85980945-05c1-455e-a702-9accf0ca790a service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:55.465004 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85980945-05c1-455e-a702-9accf0ca790a service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:55.465280 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85980945-05c1-455e-a702-9accf0ca790a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:55.521044 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f2e8f0-75ea-4af5-a364-f31a46d332e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:55.521715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2e8f0-75ea-4af5-a364-f31a46d332e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:55.522020 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2e8f0-75ea-4af5-a364-f31a46d332e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:55.522321 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2e8f0-75ea-4af5-a364-f31a46d332e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:55.523012 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f2e8f0-75ea-4af5-a364-f31a46d332e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:55.524925 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f2e8f0-75ea-4af5-a364-f31a46d332e0 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:55.530535 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85980945-05c1-455e-a702-9accf0ca790a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:55.532660 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85980945-05c1-455e-a702-9accf0ca790a service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:55.544596 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1029/2065] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:55 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:38:55.545368 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85980945-05c1-455e-a702-9accf0ca790a service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:55.547141 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-85980945-05c1-455e-a702-9accf0ca790a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:55.554161 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1037/2066] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:55 2022] POST /identity/v3/users => generated 286 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:55.555544 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f2e8f0-75ea-4af5-a364-f31a46d332e0 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:55.558511 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-28f2e8f0-75ea-4af5-a364-f31a46d332e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:55.560050 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85980945-05c1-455e-a702-9accf0ca790a service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:55.564058 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0363f9-ceab-4633-9350-005c2f4dacd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:55.564775 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0363f9-ceab-4633-9350-005c2f4dacd2 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:55.565129 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0363f9-ceab-4633-9350-005c2f4dacd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:55.565516 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0363f9-ceab-4633-9350-005c2f4dacd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:55.571582 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f2e8f0-75ea-4af5-a364-f31a46d332e0 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:55.576358 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d5307-f5ef-4225-86dc-22e0ab64d603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:55.577287 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d5307-f5ef-4225-86dc-22e0ab64d603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:55.577644 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d5307-f5ef-4225-86dc-22e0ab64d603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:55.578014 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d5307-f5ef-4225-86dc-22e0ab64d603 admin admin] PATH_INFO: `/v3/projects/3b247a30aa2841a698be29e5aaf92edc` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:55.579290 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8d5307-f5ef-4225-86dc-22e0ab64d603 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b247a30aa2841a698be29e5aaf92edc)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:55.581326 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8d5307-f5ef-4225-86dc-22e0ab64d603 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:55.628906 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0363f9-ceab-4633-9350-005c2f4dacd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:55.632200 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0363f9-ceab-4633-9350-005c2f4dacd2 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:55.641954 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1030/2067] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:55 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:38:55.642896 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0363f9-ceab-4633-9350-005c2f4dacd2 service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:55.643674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-eb8d5307-f5ef-4225-86dc-22e0ab64d603 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b247a30aa2841a698be29e5aaf92edc', 'request_id': 'req-eb8d5307-f5ef-4225-86dc-22e0ab64d603'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:55.654783 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-eb8d5307-f5ef-4225-86dc-22e0ab64d603 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b247a30aa2841a698be29e5aaf92edc', 'request_id': 'req-eb8d5307-f5ef-4225-86dc-22e0ab64d603'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:55.655288 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0363f9-ceab-4633-9350-005c2f4dacd2 service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:55.662736 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-eb8d5307-f5ef-4225-86dc-22e0ab64d603 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b247a30aa2841a698be29e5aaf92edc', 'request_id': 'req-eb8d5307-f5ef-4225-86dc-22e0ab64d603'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:55.675577 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-eb8d5307-f5ef-4225-86dc-22e0ab64d603 admin admin] The token cache is being invalidate because project 3b247a30aa2841a698be29e5aaf92edc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:55.675864 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-eb8d5307-f5ef-4225-86dc-22e0ab64d603 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:55.676768 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-eb8d5307-f5ef-4225-86dc-22e0ab64d603 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:55.678602 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1038/2068] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:55 2022] DELETE /identity/v3/projects/3b247a30aa2841a698be29e5aaf92edc => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:55.679146 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d5307-f5ef-4225-86dc-22e0ab64d603 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:55.680871 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-eb8d5307-f5ef-4225-86dc-22e0ab64d603 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:55.716192 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49800b5-3767-48bd-8662-72b50e90519e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:55.716780 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49800b5-3767-48bd-8662-72b50e90519e admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:55.717063 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49800b5-3767-48bd-8662-72b50e90519e admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:55.717340 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49800b5-3767-48bd-8662-72b50e90519e admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:55.718068 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49800b5-3767-48bd-8662-72b50e90519e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:55.719759 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49800b5-3767-48bd-8662-72b50e90519e admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:55.721865 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:55.734033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1031/2069] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:55.734033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49800b5-3767-48bd-8662-72b50e90519e admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:55.734033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b49800b5-3767-48bd-8662-72b50e90519e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:55.742838 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:55.747305 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49800b5-3767-48bd-8662-72b50e90519e admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:55.751176 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d5307-f5ef-4225-86dc-22e0ab64d603 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:55.753960 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e6dd4-e270-44ef-8a0f-fb9ff526ca4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:55.754898 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e6dd4-e270-44ef-8a0f-fb9ff526ca4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:55.755309 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e6dd4-e270-44ef-8a0f-fb9ff526ca4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:55.755581 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e6dd4-e270-44ef-8a0f-fb9ff526ca4f admin admin] PATH_INFO: `/v3/users/c9fb7e763a3445bcb2293e447548d8f4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:55.756814 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e6dd4-e270-44ef-8a0f-fb9ff526ca4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9fb7e763a3445bcb2293e447548d8f4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:55.757895 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce7d24-12a1-4ef9-b923-220c5b19fa4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:55.758937 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e6dd4-e270-44ef-8a0f-fb9ff526ca4f admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:55.759912 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce7d24-12a1-4ef9-b923-220c5b19fa4b service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:55.760327 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce7d24-12a1-4ef9-b923-220c5b19fa4b service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:55.760676 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce7d24-12a1-4ef9-b923-220c5b19fa4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:55.762864 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-e1ce7d24-12a1-4ef9-b923-220c5b19fa4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:55.815323 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-361e6dd4-e270-44ef-8a0f-fb9ff526ca4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9fb7e763a3445bcb2293e447548d8f4'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:55.815958 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-361e6dd4-e270-44ef-8a0f-fb9ff526ca4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9fb7e763a3445bcb2293e447548d8f4'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:55.816699 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-361e6dd4-e270-44ef-8a0f-fb9ff526ca4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9fb7e763a3445bcb2293e447548d8f4'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:55.829359 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce7d24-12a1-4ef9-b923-220c5b19fa4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:55.831084 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce7d24-12a1-4ef9-b923-220c5b19fa4b service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:55.833299 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-361e6dd4-e270-44ef-8a0f-fb9ff526ca4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9fb7e763a3445bcb2293e447548d8f4'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:55.840942 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1039/2070] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:38:55.841831 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce7d24-12a1-4ef9-b923-220c5b19fa4b service cinder] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:55.842028 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-361e6dd4-e270-44ef-8a0f-fb9ff526ca4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9fb7e763a3445bcb2293e447548d8f4'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:55.843263 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-e1ce7d24-12a1-4ef9-b923-220c5b19fa4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:55.860055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1032/2071] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:38:55 2022] DELETE /identity/v3/users/c9fb7e763a3445bcb2293e447548d8f4 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:55.869667 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce7d24-12a1-4ef9-b923-220c5b19fa4b service cinder] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:55.964312 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efe210e-b8f6-4243-adff-5b377304e83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:55.965331 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe210e-b8f6-4243-adff-5b377304e83c admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:55.965988 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe210e-b8f6-4243-adff-5b377304e83c admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:55.966515 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe210e-b8f6-4243-adff-5b377304e83c admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:55.967493 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efe210e-b8f6-4243-adff-5b377304e83c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:55.970745 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efe210e-b8f6-4243-adff-5b377304e83c admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:56.004677 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1040/2072] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:56.005920 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e6dd4-e270-44ef-8a0f-fb9ff526ca4f admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:56.007810 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-361e6dd4-e270-44ef-8a0f-fb9ff526ca4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:56.037584 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e6dd4-e270-44ef-8a0f-fb9ff526ca4f admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:56.065067 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20643505-9ac6-4920-99a3-3b37070135ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:56.066075 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20643505-9ac6-4920-99a3-3b37070135ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:56.066995 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20643505-9ac6-4920-99a3-3b37070135ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:56.067292 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20643505-9ac6-4920-99a3-3b37070135ca admin admin] PATH_INFO: `/v3/projects/431ec20d7b2e4acc96d76e5a03fa98e1/users/1238326ce5064afa94f2f4cbd5093113/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:56.108294 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20643505-9ac6-4920-99a3-3b37070135ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=431ec20d7b2e4acc96d76e5a03fa98e1, user_id=1238326ce5064afa94f2f4cbd5093113, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:56.119753 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20643505-9ac6-4920-99a3-3b37070135ca admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:56.154712 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1033/2073] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:56 2022] PUT /identity/v3/projects/431ec20d7b2e4acc96d76e5a03fa98e1/users/1238326ce5064afa94f2f4cbd5093113/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:56.166523 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7513828-5fef-4145-bb1c-21be08126522 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:56.167738 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7513828-5fef-4145-bb1c-21be08126522 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:56.167738 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7513828-5fef-4145-bb1c-21be08126522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:56.204273 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-e7513828-5fef-4145-bb1c-21be08126522 None None] MFA Rules not processed for user `1238326ce5064afa94f2f4cbd5093113`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:56.205813 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-e7513828-5fef-4145-bb1c-21be08126522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:56.287953 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1041/2074] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:56.300440 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20643505-9ac6-4920-99a3-3b37070135ca admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:56.302234 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-20643505-9ac6-4920-99a3-3b37070135ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:56.321140 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7513828-5fef-4145-bb1c-21be08126522 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:56.323077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-e7513828-5fef-4145-bb1c-21be08126522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:56.370584 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20643505-9ac6-4920-99a3-3b37070135ca admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:56.373061 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:56.375991 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a0d86-63a7-4a4f-ae01-97de56329e43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:56.376898 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a0d86-63a7-4a4f-ae01-97de56329e43 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:56.377317 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a0d86-63a7-4a4f-ae01-97de56329e43 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:56.378382 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a0d86-63a7-4a4f-ae01-97de56329e43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:56.390575 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7a0d86-63a7-4a4f-ae01-97de56329e43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:56.392298 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7a0d86-63a7-4a4f-ae01-97de56329e43 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:56.402437 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1034/2075] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:38:56.403330 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a0d86-63a7-4a4f-ae01-97de56329e43 service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:56.404871 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0c7a0d86-63a7-4a4f-ae01-97de56329e43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:56.417833 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7513828-5fef-4145-bb1c-21be08126522 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:56.423449 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb263d-2827-4ef6-8597-f19fa105bcd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:56.424433 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb263d-2827-4ef6-8597-f19fa105bcd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:56.424714 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb263d-2827-4ef6-8597-f19fa105bcd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:56.424981 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb263d-2827-4ef6-8597-f19fa105bcd7 admin admin] PATH_INFO: `/v3/users/14970397a0ec49c98c4e82d994a56858` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:56.426462 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cb263d-2827-4ef6-8597-f19fa105bcd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14970397a0ec49c98c4e82d994a56858)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:56.428844 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cb263d-2827-4ef6-8597-f19fa105bcd7 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:56.429973 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a0d86-63a7-4a4f-ae01-97de56329e43 service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:56.436806 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cef438-a272-4bdc-a14d-3099514283c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:56.437698 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cef438-a272-4bdc-a14d-3099514283c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:56.437973 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cef438-a272-4bdc-a14d-3099514283c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:56.438371 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cef438-a272-4bdc-a14d-3099514283c8 admin admin] PATH_INFO: `/v3/projects/04fb2007c7df4df39fded5d039b478c9` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:56.439528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cef438-a272-4bdc-a14d-3099514283c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04fb2007c7df4df39fded5d039b478c9)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:56.441434 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cef438-a272-4bdc-a14d-3099514283c8 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:56.480827 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-88cb263d-2827-4ef6-8597-f19fa105bcd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14970397a0ec49c98c4e82d994a56858'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:56.481437 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-88cb263d-2827-4ef6-8597-f19fa105bcd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14970397a0ec49c98c4e82d994a56858'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:56.488842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-88cb263d-2827-4ef6-8597-f19fa105bcd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14970397a0ec49c98c4e82d994a56858'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:56.489619 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-88cb263d-2827-4ef6-8597-f19fa105bcd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14970397a0ec49c98c4e82d994a56858'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:56.489759 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:56.494521 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-67cef438-a272-4bdc-a14d-3099514283c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04fb2007c7df4df39fded5d039b478c9', 'request_id': 'req-67cef438-a272-4bdc-a14d-3099514283c8'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:56.497484 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-88cb263d-2827-4ef6-8597-f19fa105bcd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14970397a0ec49c98c4e82d994a56858'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:56.503886 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-67cef438-a272-4bdc-a14d-3099514283c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04fb2007c7df4df39fded5d039b478c9', 'request_id': 'req-67cef438-a272-4bdc-a14d-3099514283c8'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:56.510621 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-67cef438-a272-4bdc-a14d-3099514283c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04fb2007c7df4df39fded5d039b478c9', 'request_id': 'req-67cef438-a272-4bdc-a14d-3099514283c8'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:56.519410 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1042/2076] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:38:56 2022] DELETE /identity/v3/users/14970397a0ec49c98c4e82d994a56858 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:56.519410 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb263d-2827-4ef6-8597-f19fa105bcd7 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:56.519410 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-88cb263d-2827-4ef6-8597-f19fa105bcd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:56.524679 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-67cef438-a272-4bdc-a14d-3099514283c8 admin admin] The token cache is being invalidate because project 04fb2007c7df4df39fded5d039b478c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:56.524967 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-67cef438-a272-4bdc-a14d-3099514283c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:56.525770 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-67cef438-a272-4bdc-a14d-3099514283c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:56.527943 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1035/2077] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:56 2022] DELETE /identity/v3/projects/04fb2007c7df4df39fded5d039b478c9 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:56.532284 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e383e3-4bdb-4f6f-b75f-704e42780132 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:56.532284 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e383e3-4bdb-4f6f-b75f-704e42780132 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:56.532284 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e383e3-4bdb-4f6f-b75f-704e42780132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:56.547830 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-29e383e3-4bdb-4f6f-b75f-704e42780132 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:56.549112 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-29e383e3-4bdb-4f6f-b75f-704e42780132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:56.588488 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb263d-2827-4ef6-8597-f19fa105bcd7 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:56.593446 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cca63d6-f8c4-40e9-ae08-9aaa8c361404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:56.594349 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca63d6-f8c4-40e9-ae08-9aaa8c361404 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:56.594637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca63d6-f8c4-40e9-ae08-9aaa8c361404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:56.594914 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca63d6-f8c4-40e9-ae08-9aaa8c361404 admin admin] PATH_INFO: `/v3/users/1238326ce5064afa94f2f4cbd5093113` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:56.596276 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cca63d6-f8c4-40e9-ae08-9aaa8c361404 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1238326ce5064afa94f2f4cbd5093113)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:56.598235 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cca63d6-f8c4-40e9-ae08-9aaa8c361404 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:56.609659 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1036/2078] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:56.611062 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1d599-8039-4761-872b-e722103db688 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:56.611356 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1d599-8039-4761-872b-e722103db688 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:56.611633 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1d599-8039-4761-872b-e722103db688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:56.630572 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-10f1d599-8039-4761-872b-e722103db688 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:56.632031 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-10f1d599-8039-4761-872b-e722103db688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:56.641054 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1037/2079] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:56.646980 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f1d599-8039-4761-872b-e722103db688 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:56.647774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-5cca63d6-f8c4-40e9-ae08-9aaa8c361404 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1238326ce5064afa94f2f4cbd5093113'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:56.648502 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-5cca63d6-f8c4-40e9-ae08-9aaa8c361404 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1238326ce5064afa94f2f4cbd5093113'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:56.648636 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-10f1d599-8039-4761-872b-e722103db688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:56.650259 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:56.658443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-5cca63d6-f8c4-40e9-ae08-9aaa8c361404 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1238326ce5064afa94f2f4cbd5093113'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:56.659303 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-5cca63d6-f8c4-40e9-ae08-9aaa8c361404 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1238326ce5064afa94f2f4cbd5093113'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:56.666594 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-5cca63d6-f8c4-40e9-ae08-9aaa8c361404 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1238326ce5064afa94f2f4cbd5093113'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:56.687702 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1043/2080] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:38:56 2022] DELETE /identity/v3/users/1238326ce5064afa94f2f4cbd5093113 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:56.689086 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472aab3-9afa-4ad2-bb82-01dc53d26455 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:56.689378 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472aab3-9afa-4ad2-bb82-01dc53d26455 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:56.689659 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472aab3-9afa-4ad2-bb82-01dc53d26455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:56.707784 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-5472aab3-9afa-4ad2-bb82-01dc53d26455 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:56.708482 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f1d599-8039-4761-872b-e722103db688 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:56.708970 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-5472aab3-9afa-4ad2-bb82-01dc53d26455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:56.756753 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12757fff-1018-4feb-8a34-7908c6135426 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:56.757407 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12757fff-1018-4feb-8a34-7908c6135426 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:56.757769 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12757fff-1018-4feb-8a34-7908c6135426 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:56.758054 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12757fff-1018-4feb-8a34-7908c6135426 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:56.760254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-12757fff-1018-4feb-8a34-7908c6135426 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:56.768329 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1044/2081] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:56.769779 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e22f3-cd0e-4ec4-8bb1-34c8f4cd5561 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:56.770686 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e22f3-cd0e-4ec4-8bb1-34c8f4cd5561 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:56.770973 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e22f3-cd0e-4ec4-8bb1-34c8f4cd5561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:56.772500 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12757fff-1018-4feb-8a34-7908c6135426 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:56.773963 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12757fff-1018-4feb-8a34-7908c6135426 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:56.783523 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1038/2082] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:38:56.784305 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12757fff-1018-4feb-8a34-7908c6135426 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:56.788308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-230e22f3-cd0e-4ec4-8bb1-34c8f4cd5561 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:56.789512 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-230e22f3-cd0e-4ec4-8bb1-34c8f4cd5561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:56.793764 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12757fff-1018-4feb-8a34-7908c6135426 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:56.798177 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1045/2083] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:56.798354 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319d0eea-a11c-4b36-ac4a-9913f4e8d8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:56.799000 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d0eea-a11c-4b36-ac4a-9913f4e8d8f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:56.799277 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d0eea-a11c-4b36-ac4a-9913f4e8d8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:56.799552 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d0eea-a11c-4b36-ac4a-9913f4e8d8f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:56.800352 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319d0eea-a11c-4b36-ac4a-9913f4e8d8f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:38:56.800663 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319d0eea-a11c-4b36-ac4a-9913f4e8d8f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:56.802273 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319d0eea-a11c-4b36-ac4a-9913f4e8d8f5 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:56.809769 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e22f3-cd0e-4ec4-8bb1-34c8f4cd5561 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:56.812041 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-230e22f3-cd0e-4ec4-8bb1-34c8f4cd5561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:56.818648 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1039/2084] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:38:56.819540 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319d0eea-a11c-4b36-ac4a-9913f4e8d8f5 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:56.829604 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e22f3-cd0e-4ec4-8bb1-34c8f4cd5561 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:56.831024 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319d0eea-a11c-4b36-ac4a-9913f4e8d8f5 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:56.835063 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6cd40-ba5e-4413-81c4-99df50a2f222 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:56.836131 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0305cd-64b5-4bb2-8bbc-b042138e271c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:56.836640 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0305cd-64b5-4bb2-8bbc-b042138e271c admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:56.836878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6cd40-ba5e-4413-81c4-99df50a2f222 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:56.836990 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0305cd-64b5-4bb2-8bbc-b042138e271c admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:56.837183 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0305cd-64b5-4bb2-8bbc-b042138e271c admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:56.837630 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6cd40-ba5e-4413-81c4-99df50a2f222 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:56.837976 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0305cd-64b5-4bb2-8bbc-b042138e271c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:56.838330 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6cd40-ba5e-4413-81c4-99df50a2f222 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:56.840771 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0305cd-64b5-4bb2-8bbc-b042138e271c admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:56.851421 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:56.854119 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d6cd40-ba5e-4413-81c4-99df50a2f222 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:56.855610 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d6cd40-ba5e-4413-81c4-99df50a2f222 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:56.872959 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1040/2085] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:56 2022] POST /identity/v3/projects => generated 359 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:38:56.874275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1046/2086] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:38:56.883200 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0305cd-64b5-4bb2-8bbc-b042138e271c admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:56.964500 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0305cd-64b5-4bb2-8bbc-b042138e271c admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:56.971553 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5813ab23-f94f-4cd2-8740-685264e46b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:56.972205 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813ab23-f94f-4cd2-8740-685264e46b13 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:56.972486 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813ab23-f94f-4cd2-8740-685264e46b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:56.972762 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813ab23-f94f-4cd2-8740-685264e46b13 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:56.973457 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5813ab23-f94f-4cd2-8740-685264e46b13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:56.975470 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5813ab23-f94f-4cd2-8740-685264e46b13 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:56.998215 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1041/2087] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:56 2022] POST /identity/v3/users => generated 280 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:57.002660 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6cd40-ba5e-4413-81c4-99df50a2f222 service neutron] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:57.012832 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6cd40-ba5e-4413-81c4-99df50a2f222 service neutron] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:57.017564 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51de899-a177-4b2f-8cf5-33776c3b3b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:57.018408 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51de899-a177-4b2f-8cf5-33776c3b3b56 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:57.018690 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51de899-a177-4b2f-8cf5-33776c3b3b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:57.018964 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51de899-a177-4b2f-8cf5-33776c3b3b56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:57.019724 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51de899-a177-4b2f-8cf5-33776c3b3b56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:57.021306 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51de899-a177-4b2f-8cf5-33776c3b3b56 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:57.031889 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1047/2088] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:38:57.036006 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5813ab23-f94f-4cd2-8740-685264e46b13 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:57.046766 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5813ab23-f94f-4cd2-8740-685264e46b13 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:57.051443 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ebe7f-b66b-494b-925c-b5703e8330ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:57.052107 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ebe7f-b66b-494b-925c-b5703e8330ce admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:57.052486 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ebe7f-b66b-494b-925c-b5703e8330ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:57.052745 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ebe7f-b66b-494b-925c-b5703e8330ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:57.053477 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16ebe7f-b66b-494b-925c-b5703e8330ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:57.055233 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16ebe7f-b66b-494b-925c-b5703e8330ce admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:57.069699 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1042/2089] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:57.075170 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51de899-a177-4b2f-8cf5-33776c3b3b56 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:57.091332 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51de899-a177-4b2f-8cf5-33776c3b3b56 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:57.097110 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324fd60-6365-4b65-bbae-1369ca420cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:57.099183 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324fd60-6365-4b65-bbae-1369ca420cac admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:57.099755 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324fd60-6365-4b65-bbae-1369ca420cac admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:57.100186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324fd60-6365-4b65-bbae-1369ca420cac admin admin] PATH_INFO: `/v3/projects/97bd09c52c6143378e03a7a4ac25d7ac/users/b9cde3000c6f49a3a20a7dac555e751a/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:57.120287 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e324fd60-6365-4b65-bbae-1369ca420cac admin admin] RBAC: Authorizing `identity:create_grant(project_id=97bd09c52c6143378e03a7a4ac25d7ac, user_id=b9cde3000c6f49a3a20a7dac555e751a, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:57.124198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e324fd60-6365-4b65-bbae-1369ca420cac admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:57.149264 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1048/2090] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:57 2022] PUT /identity/v3/projects/97bd09c52c6143378e03a7a4ac25d7ac/users/b9cde3000c6f49a3a20a7dac555e751a/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:57.152784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f207c9-c76d-4f57-b477-770ee46c83a5 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:57.153905 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f207c9-c76d-4f57-b477-770ee46c83a5 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:57.155050 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f207c9-c76d-4f57-b477-770ee46c83a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:57.189856 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-f6f207c9-c76d-4f57-b477-770ee46c83a5 None None] MFA Rules not processed for user `b9cde3000c6f49a3a20a7dac555e751a`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:57.191261 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f6f207c9-c76d-4f57-b477-770ee46c83a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:57.222493 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324fd60-6365-4b65-bbae-1369ca420cac admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:57.224962 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-e324fd60-6365-4b65-bbae-1369ca420cac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:57.322778 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324fd60-6365-4b65-bbae-1369ca420cac admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:57.328826 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98606929-d7b1-426c-aeb9-3752ed727d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:57.329880 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98606929-d7b1-426c-aeb9-3752ed727d8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:57.330199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98606929-d7b1-426c-aeb9-3752ed727d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:57.330494 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98606929-d7b1-426c-aeb9-3752ed727d8d admin admin] PATH_INFO: `/v3/projects/45247c5e973e47b2be087d24a818400b` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:57.332187 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98606929-d7b1-426c-aeb9-3752ed727d8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=45247c5e973e47b2be087d24a818400b)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:57.332439 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1043/2091] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:57 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:57.334288 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98606929-d7b1-426c-aeb9-3752ed727d8d admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:57.342375 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f207c9-c76d-4f57-b477-770ee46c83a5 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:57.352697 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f6f207c9-c76d-4f57-b477-770ee46c83a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:57.361751 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:57.370312 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f207c9-c76d-4f57-b477-770ee46c83a5 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:57.376667 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e989e-1070-417e-a0d0-74714b891578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:57.377730 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e989e-1070-417e-a0d0-74714b891578 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:57.378035 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e989e-1070-417e-a0d0-74714b891578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:57.378348 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e989e-1070-417e-a0d0-74714b891578 admin admin] PATH_INFO: `/v3/projects/431ec20d7b2e4acc96d76e5a03fa98e1` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:57.380146 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e989e-1070-417e-a0d0-74714b891578 admin admin] RBAC: Authorizing `identity:delete_project(project_id=431ec20d7b2e4acc96d76e5a03fa98e1)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:57.382400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e989e-1070-417e-a0d0-74714b891578 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:57.407385 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-98606929-d7b1-426c-aeb9-3752ed727d8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45247c5e973e47b2be087d24a818400b', 'request_id': 'req-98606929-d7b1-426c-aeb9-3752ed727d8d'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:57.417421 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-98606929-d7b1-426c-aeb9-3752ed727d8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45247c5e973e47b2be087d24a818400b', 'request_id': 'req-98606929-d7b1-426c-aeb9-3752ed727d8d'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:57.424651 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-98606929-d7b1-426c-aeb9-3752ed727d8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45247c5e973e47b2be087d24a818400b', 'request_id': 'req-98606929-d7b1-426c-aeb9-3752ed727d8d'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:57.441385 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-98606929-d7b1-426c-aeb9-3752ed727d8d admin admin] The token cache is being invalidate because project 45247c5e973e47b2be087d24a818400b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:57.441757 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-98606929-d7b1-426c-aeb9-3752ed727d8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:57.443453 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-98606929-d7b1-426c-aeb9-3752ed727d8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:57.444005 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-099e989e-1070-417e-a0d0-74714b891578 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '431ec20d7b2e4acc96d76e5a03fa98e1', 'request_id': 'req-099e989e-1070-417e-a0d0-74714b891578'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:57.445151 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1049/2092] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:57 2022] DELETE /identity/v3/projects/45247c5e973e47b2be087d24a818400b => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:38:57.446250 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98606929-d7b1-426c-aeb9-3752ed727d8d admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:57.447528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:57.449276 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-98606929-d7b1-426c-aeb9-3752ed727d8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:57.454944 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-099e989e-1070-417e-a0d0-74714b891578 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '431ec20d7b2e4acc96d76e5a03fa98e1', 'request_id': 'req-099e989e-1070-417e-a0d0-74714b891578'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:57.465774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-099e989e-1070-417e-a0d0-74714b891578 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '431ec20d7b2e4acc96d76e5a03fa98e1', 'request_id': 'req-099e989e-1070-417e-a0d0-74714b891578'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:57.478192 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-099e989e-1070-417e-a0d0-74714b891578 admin admin] The token cache is being invalidate because project 431ec20d7b2e4acc96d76e5a03fa98e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:57.478562 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-099e989e-1070-417e-a0d0-74714b891578 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:57.479855 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-099e989e-1070-417e-a0d0-74714b891578 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:57.482026 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1044/2093] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:57 2022] DELETE /identity/v3/projects/431ec20d7b2e4acc96d76e5a03fa98e1 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:57.483013 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e989e-1070-417e-a0d0-74714b891578 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:57.484864 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-099e989e-1070-417e-a0d0-74714b891578 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:57.523345 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98606929-d7b1-426c-aeb9-3752ed727d8d admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:57.568879 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e989e-1070-417e-a0d0-74714b891578 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:57.575397 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a327f-bb86-453a-bda9-7621af26b21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:57.576398 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a327f-bb86-453a-bda9-7621af26b21e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:57.576687 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a327f-bb86-453a-bda9-7621af26b21e admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:57.576963 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a327f-bb86-453a-bda9-7621af26b21e admin admin] PATH_INFO: `/v3/users/0faeee214e1544bfba1e9c8c12cf05d0` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:57.577468 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb1c37-1741-4808-ad41-ae8e15f5d96c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:57.578149 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb1c37-1741-4808-ad41-ae8e15f5d96c service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:57.578261 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a327f-bb86-453a-bda9-7621af26b21e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0faeee214e1544bfba1e9c8c12cf05d0)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:57.578437 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb1c37-1741-4808-ad41-ae8e15f5d96c service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:57.578715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb1c37-1741-4808-ad41-ae8e15f5d96c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:57.580151 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a327f-bb86-453a-bda9-7621af26b21e admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:57.580564 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-63fb1c37-1741-4808-ad41-ae8e15f5d96c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:57.627720 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-9e8a327f-bb86-453a-bda9-7621af26b21e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0faeee214e1544bfba1e9c8c12cf05d0'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:57.628436 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-9e8a327f-bb86-453a-bda9-7621af26b21e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0faeee214e1544bfba1e9c8c12cf05d0'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:57.629055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-9e8a327f-bb86-453a-bda9-7621af26b21e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0faeee214e1544bfba1e9c8c12cf05d0'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:57.641575 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-9e8a327f-bb86-453a-bda9-7621af26b21e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0faeee214e1544bfba1e9c8c12cf05d0'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:57.643019 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fb1c37-1741-4808-ad41-ae8e15f5d96c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:57.644472 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fb1c37-1741-4808-ad41-ae8e15f5d96c service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:57.648788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-9e8a327f-bb86-453a-bda9-7621af26b21e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0faeee214e1544bfba1e9c8c12cf05d0'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:57.654224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1050/2094] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:57 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:38:57.655646 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288e149-ad0e-46a3-a896-55dbebc1b0f0 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:57.655941 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288e149-ad0e-46a3-a896-55dbebc1b0f0 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:57.656295 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288e149-ad0e-46a3-a896-55dbebc1b0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:57.665231 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1045/2095] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:38:57 2022] DELETE /identity/v3/users/0faeee214e1544bfba1e9c8c12cf05d0 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:57.669807 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a327f-bb86-453a-bda9-7621af26b21e admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:57.671313 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-9e8a327f-bb86-453a-bda9-7621af26b21e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:57.688753 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-4288e149-ad0e-46a3-a896-55dbebc1b0f0 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:57.689959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-4288e149-ad0e-46a3-a896-55dbebc1b0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:57.696339 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:57.746229 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a327f-bb86-453a-bda9-7621af26b21e admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:57.753465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da7ac4d-cfcd-4ae6-8944-8ab8c143a526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:57.754528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7ac4d-cfcd-4ae6-8944-8ab8c143a526 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:57.754906 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7ac4d-cfcd-4ae6-8944-8ab8c143a526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:57.755724 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7ac4d-cfcd-4ae6-8944-8ab8c143a526 admin admin] PATH_INFO: `/v3/users/b9cde3000c6f49a3a20a7dac555e751a` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:57.756983 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da7ac4d-cfcd-4ae6-8944-8ab8c143a526 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9cde3000c6f49a3a20a7dac555e751a)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:57.758954 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da7ac4d-cfcd-4ae6-8944-8ab8c143a526 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:57.762458 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:57.766944 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1051/2096] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:57.777979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566c214-809f-41f8-a872-40a6f6f0830a None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:57.778296 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566c214-809f-41f8-a872-40a6f6f0830a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:57.778606 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566c214-809f-41f8-a872-40a6f6f0830a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:57.798075 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-1566c214-809f-41f8-a872-40a6f6f0830a None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:57.799700 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-1566c214-809f-41f8-a872-40a6f6f0830a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:57.818559 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1da7ac4d-cfcd-4ae6-8944-8ab8c143a526 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9cde3000c6f49a3a20a7dac555e751a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:57.819348 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1da7ac4d-cfcd-4ae6-8944-8ab8c143a526 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9cde3000c6f49a3a20a7dac555e751a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:57.820546 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1da7ac4d-cfcd-4ae6-8944-8ab8c143a526 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9cde3000c6f49a3a20a7dac555e751a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:57.826935 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1052/2097] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:57.827339 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1566c214-809f-41f8-a872-40a6f6f0830a None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:57.836894 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1566c214-809f-41f8-a872-40a6f6f0830a None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:57.839814 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1da7ac4d-cfcd-4ae6-8944-8ab8c143a526 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9cde3000c6f49a3a20a7dac555e751a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:57.841947 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18dd66c-c974-460d-a006-8fab9723c72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:57.842561 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18dd66c-c974-460d-a006-8fab9723c72b admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:57.842847 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18dd66c-c974-460d-a006-8fab9723c72b admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:57.843125 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18dd66c-c974-460d-a006-8fab9723c72b admin admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:57.843860 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18dd66c-c974-460d-a006-8fab9723c72b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:38:57.844171 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18dd66c-c974-460d-a006-8fab9723c72b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:57.845800 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18dd66c-c974-460d-a006-8fab9723c72b admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:57.848123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1da7ac4d-cfcd-4ae6-8944-8ab8c143a526 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9cde3000c6f49a3a20a7dac555e751a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:57.859313 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1053/2098] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:38:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:38:57.863705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18dd66c-c974-460d-a006-8fab9723c72b admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:57.865205 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1046/2099] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:38:57 2022] DELETE /identity/v3/users/b9cde3000c6f49a3a20a7dac555e751a => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:57.865431 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-f18dd66c-c974-460d-a006-8fab9723c72b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:57.869479 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f374590-1d65-471c-9492-5de61fdbc0b4 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:57.870514 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f374590-1d65-471c-9492-5de61fdbc0b4 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:57.870860 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f374590-1d65-471c-9492-5de61fdbc0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:57.881616 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18dd66c-c974-460d-a006-8fab9723c72b admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:57.891048 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-8f374590-1d65-471c-9492-5de61fdbc0b4 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:57.892258 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8f374590-1d65-471c-9492-5de61fdbc0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:57.945601 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66f9f6-504e-4626-a7e0-307491d3d144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:57.946179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66f9f6-504e-4626-a7e0-307491d3d144 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:57.946468 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66f9f6-504e-4626-a7e0-307491d3d144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:57.946742 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66f9f6-504e-4626-a7e0-307491d3d144 admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:57.947457 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b66f9f6-504e-4626-a7e0-307491d3d144 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:57.949975 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b66f9f6-504e-4626-a7e0-307491d3d144 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:57.964027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1047/2100] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:38:57.970501 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f374590-1d65-471c-9492-5de61fdbc0b4 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:57.972607 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8f374590-1d65-471c-9492-5de61fdbc0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:57.988016 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f374590-1d65-471c-9492-5de61fdbc0b4 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:57.994269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1054/2101] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:57 2022] POST /identity/v3/projects => generated 369 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:38:57.994269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66f9f6-504e-4626-a7e0-307491d3d144 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:57.994269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-9b66f9f6-504e-4626-a7e0-307491d3d144 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:57.997367 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3109ac-7c26-4614-9811-e261514d6a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:57.998480 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3109ac-7c26-4614-9811-e261514d6a1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:57.998933 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3109ac-7c26-4614-9811-e261514d6a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:57.999458 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3109ac-7c26-4614-9811-e261514d6a1c admin admin] PATH_INFO: `/v3/projects/592f1a334ef34db6ad1fdea5154fcc26` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:58.001251 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3109ac-7c26-4614-9811-e261514d6a1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=592f1a334ef34db6ad1fdea5154fcc26)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:58.004063 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3109ac-7c26-4614-9811-e261514d6a1c admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:58.052794 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0d3109ac-7c26-4614-9811-e261514d6a1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '592f1a334ef34db6ad1fdea5154fcc26', 'request_id': 'req-0d3109ac-7c26-4614-9811-e261514d6a1c'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:58.059751 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66f9f6-504e-4626-a7e0-307491d3d144 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:58.060727 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0d3109ac-7c26-4614-9811-e261514d6a1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '592f1a334ef34db6ad1fdea5154fcc26', 'request_id': 'req-0d3109ac-7c26-4614-9811-e261514d6a1c'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:58.067406 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0d3109ac-7c26-4614-9811-e261514d6a1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '592f1a334ef34db6ad1fdea5154fcc26', 'request_id': 'req-0d3109ac-7c26-4614-9811-e261514d6a1c'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:58.078275 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0d3109ac-7c26-4614-9811-e261514d6a1c admin admin] The token cache is being invalidate because project 592f1a334ef34db6ad1fdea5154fcc26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:58.078564 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0d3109ac-7c26-4614-9811-e261514d6a1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:58.079217 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0d3109ac-7c26-4614-9811-e261514d6a1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:58.080759 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1048/2102] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:57 2022] DELETE /identity/v3/projects/592f1a334ef34db6ad1fdea5154fcc26 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:38:58.081497 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3109ac-7c26-4614-9811-e261514d6a1c admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:58.085726 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0d3109ac-7c26-4614-9811-e261514d6a1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:58.090748 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:58.094022 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:58.115333 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65902243-5ca3-4bc5-adce-22cfc85d252d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:58.115990 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65902243-5ca3-4bc5-adce-22cfc85d252d service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:58.116277 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65902243-5ca3-4bc5-adce-22cfc85d252d service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:58.116556 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65902243-5ca3-4bc5-adce-22cfc85d252d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:58.118415 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-65902243-5ca3-4bc5-adce-22cfc85d252d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:58.157034 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3109ac-7c26-4614-9811-e261514d6a1c admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:58.162318 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c7d92-9eba-4fcf-b0df-baf4ce872303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:58.163008 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c7d92-9eba-4fcf-b0df-baf4ce872303 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:58.163290 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c7d92-9eba-4fcf-b0df-baf4ce872303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:58.163582 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c7d92-9eba-4fcf-b0df-baf4ce872303 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:58.164250 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c7d92-9eba-4fcf-b0df-baf4ce872303 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:58.166225 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c7d92-9eba-4fcf-b0df-baf4ce872303 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:58.186923 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1049/2103] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:58 2022] POST /identity/v3/users => generated 285 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:58.187991 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65902243-5ca3-4bc5-adce-22cfc85d252d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:58.188134 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46086fa2-6b1e-42f0-9cb7-58d49ec20ced None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:58.188423 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46086fa2-6b1e-42f0-9cb7-58d49ec20ced None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:58.188696 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46086fa2-6b1e-42f0-9cb7-58d49ec20ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:58.189721 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65902243-5ca3-4bc5-adce-22cfc85d252d service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:58.199806 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1055/2104] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:38:58.200567 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65902243-5ca3-4bc5-adce-22cfc85d252d service neutron] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:58.206453 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-46086fa2-6b1e-42f0-9cb7-58d49ec20ced None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:58.207645 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-46086fa2-6b1e-42f0-9cb7-58d49ec20ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:58.209700 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65902243-5ca3-4bc5-adce-22cfc85d252d service neutron] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:58.214462 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddfb01c-9aea-4e65-abe3-0777d8971350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:58.215246 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddfb01c-9aea-4e65-abe3-0777d8971350 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:58.215606 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddfb01c-9aea-4e65-abe3-0777d8971350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:58.215947 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddfb01c-9aea-4e65-abe3-0777d8971350 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:58.216677 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1050/2105] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:58.216817 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddfb01c-9aea-4e65-abe3-0777d8971350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:58.218489 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddfb01c-9aea-4e65-abe3-0777d8971350 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:58.223615 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81add863-08a9-4b8e-a725-5ac1f1f8c3b3 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:58.223905 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81add863-08a9-4b8e-a725-5ac1f1f8c3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:58.224181 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81add863-08a9-4b8e-a725-5ac1f1f8c3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:58.229019 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1056/2106] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:58.233225 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddfb01c-9aea-4e65-abe3-0777d8971350 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:58.243013 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddfb01c-9aea-4e65-abe3-0777d8971350 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:58.246362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-81add863-08a9-4b8e-a725-5ac1f1f8c3b3 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:58.247608 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-81add863-08a9-4b8e-a725-5ac1f1f8c3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:58.250449 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9455e2a6-829d-44c2-b491-595548579519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:58.251117 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455e2a6-829d-44c2-b491-595548579519 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:58.251401 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455e2a6-829d-44c2-b491-595548579519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:58.251672 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455e2a6-829d-44c2-b491-595548579519 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:58.252404 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9455e2a6-829d-44c2-b491-595548579519 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:58.254022 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9455e2a6-829d-44c2-b491-595548579519 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:58.257674 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1051/2107] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:38:58.262019 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81add863-08a9-4b8e-a725-5ac1f1f8c3b3 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:58.268527 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1057/2108] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:38:58.271932 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81add863-08a9-4b8e-a725-5ac1f1f8c3b3 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:58.272635 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9455e2a6-829d-44c2-b491-595548579519 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:58.277443 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596385f1-8f9c-428c-a749-9ad02010ab80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:58.278300 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596385f1-8f9c-428c-a749-9ad02010ab80 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:58.278582 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596385f1-8f9c-428c-a749-9ad02010ab80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:58.278919 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596385f1-8f9c-428c-a749-9ad02010ab80 admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:58.280005 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596385f1-8f9c-428c-a749-9ad02010ab80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:38:58.280312 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596385f1-8f9c-428c-a749-9ad02010ab80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:58.282259 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596385f1-8f9c-428c-a749-9ad02010ab80 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:58.283750 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9455e2a6-829d-44c2-b491-595548579519 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:58.289812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda451e4-f430-4c91-ac73-a992939b94de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:58.291205 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda451e4-f430-4c91-ac73-a992939b94de admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:58.292441 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda451e4-f430-4c91-ac73-a992939b94de admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:58.292916 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda451e4-f430-4c91-ac73-a992939b94de admin admin] PATH_INFO: `/v3/projects/9fb58d9251c54f00913c42a6bbf05050/users/a3cd96ddc54246629eabb2b6ea3b4dd2/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:58.299559 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1052/2109] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:38:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:38:58.300496 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596385f1-8f9c-428c-a749-9ad02010ab80 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:58.309914 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda451e4-f430-4c91-ac73-a992939b94de admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fb58d9251c54f00913c42a6bbf05050, user_id=a3cd96ddc54246629eabb2b6ea3b4dd2, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:58.312357 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596385f1-8f9c-428c-a749-9ad02010ab80 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:58.313416 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda451e4-f430-4c91-ac73-a992939b94de admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:58.317996 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d794a48d-e4fa-4e4f-8dc9-11ef97bb7011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:58.318812 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794a48d-e4fa-4e4f-8dc9-11ef97bb7011 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:58.319266 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794a48d-e4fa-4e4f-8dc9-11ef97bb7011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:58.319713 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794a48d-e4fa-4e4f-8dc9-11ef97bb7011 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:58.320559 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d794a48d-e4fa-4e4f-8dc9-11ef97bb7011 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:58.322588 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d794a48d-e4fa-4e4f-8dc9-11ef97bb7011 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:58.333693 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1058/2110] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:58 2022] PUT /identity/v3/projects/9fb58d9251c54f00913c42a6bbf05050/users/a3cd96ddc54246629eabb2b6ea3b4dd2/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:38:58.348972 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0833595-f8d7-496d-a4ee-fb04bfdb74fb None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:58.349646 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0833595-f8d7-496d-a4ee-fb04bfdb74fb None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:58.349982 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0833595-f8d7-496d-a4ee-fb04bfdb74fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:58.353146 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1053/2111] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:58 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:58.357966 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d794a48d-e4fa-4e4f-8dc9-11ef97bb7011 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:58.393889 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-c0833595-f8d7-496d-a4ee-fb04bfdb74fb None None] MFA Rules not processed for user `a3cd96ddc54246629eabb2b6ea3b4dd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:58.395278 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-c0833595-f8d7-496d-a4ee-fb04bfdb74fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:58.432733 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d794a48d-e4fa-4e4f-8dc9-11ef97bb7011 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:58.450882 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef2dc1-a74c-474c-a38e-4a16ddc79371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:58.451866 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef2dc1-a74c-474c-a38e-4a16ddc79371 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:58.452378 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef2dc1-a74c-474c-a38e-4a16ddc79371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:58.453085 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef2dc1-a74c-474c-a38e-4a16ddc79371 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:58.454003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ef2dc1-a74c-474c-a38e-4a16ddc79371 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:58.456107 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ef2dc1-a74c-474c-a38e-4a16ddc79371 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:58.478819 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1059/2112] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:58 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:38:58.480642 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1054/2113] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:58 2022] POST /identity/v3/users => generated 292 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:58.486925 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0833595-f8d7-496d-a4ee-fb04bfdb74fb None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:58.491016 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef2dc1-a74c-474c-a38e-4a16ddc79371 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:58.492947 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c4ef2dc1-a74c-474c-a38e-4a16ddc79371 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:58.498688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0833595-f8d7-496d-a4ee-fb04bfdb74fb None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:58.504498 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dfd461-7153-4e3b-a21e-cb811e32b84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:58.506817 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dfd461-7153-4e3b-a21e-cb811e32b84d admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:58.506817 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dfd461-7153-4e3b-a21e-cb811e32b84d admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:58.506817 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dfd461-7153-4e3b-a21e-cb811e32b84d admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:58.506817 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dfd461-7153-4e3b-a21e-cb811e32b84d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:58.508203 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dfd461-7153-4e3b-a21e-cb811e32b84d admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:58.532051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1060/2114] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:58.532051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dfd461-7153-4e3b-a21e-cb811e32b84d admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:58.533526 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-f4dfd461-7153-4e3b-a21e-cb811e32b84d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:58.541537 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:58.556044 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dfd461-7153-4e3b-a21e-cb811e32b84d admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:58.561611 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b1adb-ee30-46e9-85b0-4caa0bfcfa53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:58.562600 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b1adb-ee30-46e9-85b0-4caa0bfcfa53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:58.562890 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b1adb-ee30-46e9-85b0-4caa0bfcfa53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:58.563149 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b1adb-ee30-46e9-85b0-4caa0bfcfa53 admin admin] PATH_INFO: `/v3/projects/97bd09c52c6143378e03a7a4ac25d7ac` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:58.564329 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2b1adb-ee30-46e9-85b0-4caa0bfcfa53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97bd09c52c6143378e03a7a4ac25d7ac)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:58.566294 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2b1adb-ee30-46e9-85b0-4caa0bfcfa53 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:58.572804 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef2dc1-a74c-474c-a38e-4a16ddc79371 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:58.576894 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef62f8-790c-458e-83c6-d7fb6322c2af service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfca4d302aa43249609513f35d55d7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:58.577103 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef62f8-790c-458e-83c6-d7fb6322c2af service glance] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:58.577393 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef62f8-790c-458e-83c6-d7fb6322c2af service glance] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:58.577668 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef62f8-790c-458e-83c6-d7fb6322c2af service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:58.585612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:58.587163 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef62f8-790c-458e-83c6-d7fb6322c2af service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:58.589145 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef62f8-790c-458e-83c6-d7fb6322c2af service glance] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:58.602028 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1055/2115] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:58 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:38:58.603113 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef62f8-790c-458e-83c6-d7fb6322c2af service glance] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:58.615129 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef62f8-790c-458e-83c6-d7fb6322c2af service glance] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:58.629218 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-3d2b1adb-ee30-46e9-85b0-4caa0bfcfa53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97bd09c52c6143378e03a7a4ac25d7ac', 'request_id': 'req-3d2b1adb-ee30-46e9-85b0-4caa0bfcfa53'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:58.642416 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-3d2b1adb-ee30-46e9-85b0-4caa0bfcfa53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97bd09c52c6143378e03a7a4ac25d7ac', 'request_id': 'req-3d2b1adb-ee30-46e9-85b0-4caa0bfcfa53'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:58.653695 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-3d2b1adb-ee30-46e9-85b0-4caa0bfcfa53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97bd09c52c6143378e03a7a4ac25d7ac', 'request_id': 'req-3d2b1adb-ee30-46e9-85b0-4caa0bfcfa53'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:58.675364 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-3d2b1adb-ee30-46e9-85b0-4caa0bfcfa53 admin admin] The token cache is being invalidate because project 97bd09c52c6143378e03a7a4ac25d7ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:38:58.675653 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-3d2b1adb-ee30-46e9-85b0-4caa0bfcfa53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:58.677502 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-3d2b1adb-ee30-46e9-85b0-4caa0bfcfa53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:58.683038 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1061/2116] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:58 2022] DELETE /identity/v3/projects/97bd09c52c6143378e03a7a4ac25d7ac => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:38:58.684659 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e893c-8854-4798-bdf3-aa7402a64b52 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:58.685009 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e893c-8854-4798-bdf3-aa7402a64b52 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:58.685351 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e893c-8854-4798-bdf3-aa7402a64b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:58.704504 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9eb19a-0755-4875-aa5b-4a6b705f43b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:58.705036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-143e893c-8854-4798-bdf3-aa7402a64b52 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:58.705216 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9eb19a-0755-4875-aa5b-4a6b705f43b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:58.705495 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9eb19a-0755-4875-aa5b-4a6b705f43b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:58.705771 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9eb19a-0755-4875-aa5b-4a6b705f43b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:58.706583 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-143e893c-8854-4798-bdf3-aa7402a64b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:58.707351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9eb19a-0755-4875-aa5b-4a6b705f43b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:58.709782 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9eb19a-0755-4875-aa5b-4a6b705f43b2 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:58.716701 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1062/2117] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:58.724880 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aabd5b-b622-40c2-99b2-dbad888b5d86 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:58.725339 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aabd5b-b622-40c2-99b2-dbad888b5d86 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:58.725696 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aabd5b-b622-40c2-99b2-dbad888b5d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:58.731242 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1056/2118] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:58.732992 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9eb19a-0755-4875-aa5b-4a6b705f43b2 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:58.735285 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-bc9eb19a-0755-4875-aa5b-4a6b705f43b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:58.747597 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-e7aabd5b-b622-40c2-99b2-dbad888b5d86 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:58.748839 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-e7aabd5b-b622-40c2-99b2-dbad888b5d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:58.753905 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9eb19a-0755-4875-aa5b-4a6b705f43b2 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:58.760728 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1063/2119] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:58.762219 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aabd5b-b622-40c2-99b2-dbad888b5d86 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:58.770235 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ec3987-80d8-49a0-818e-f8c364b7363b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:58.771891 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec3987-80d8-49a0-818e-f8c364b7363b admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:58.772195 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec3987-80d8-49a0-818e-f8c364b7363b admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:58.772467 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec3987-80d8-49a0-818e-f8c364b7363b admin admin] PATH_INFO: `/v3/projects/87674e7188fd4e7e9b2bd6b90a19ec99/users/7c2abbae4c894854bbbb227ad5b94444/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:58.779017 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aabd5b-b622-40c2-99b2-dbad888b5d86 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:58.783685 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db875cb7-7d85-4d6a-8d72-7f02148c44a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:58.784331 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db875cb7-7d85-4d6a-8d72-7f02148c44a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:58.784616 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db875cb7-7d85-4d6a-8d72-7f02148c44a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:58.784888 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db875cb7-7d85-4d6a-8d72-7f02148c44a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:58.785610 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db875cb7-7d85-4d6a-8d72-7f02148c44a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:38:58.785911 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db875cb7-7d85-4d6a-8d72-7f02148c44a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:58.787580 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db875cb7-7d85-4d6a-8d72-7f02148c44a9 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:58.790943 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ec3987-80d8-49a0-818e-f8c364b7363b admin admin] RBAC: Authorizing `identity:create_grant(project_id=87674e7188fd4e7e9b2bd6b90a19ec99, user_id=7c2abbae4c894854bbbb227ad5b94444, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:58.794256 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ec3987-80d8-49a0-818e-f8c364b7363b admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:58.799937 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1064/2120] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:38:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:38:58.804198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db875cb7-7d85-4d6a-8d72-7f02148c44a9 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:58.810954 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1057/2121] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:58 2022] PUT /identity/v3/projects/87674e7188fd4e7e9b2bd6b90a19ec99/users/7c2abbae4c894854bbbb227ad5b94444/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:58.823638 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db875cb7-7d85-4d6a-8d72-7f02148c44a9 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:58.825661 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb31573-7cbc-4cd5-9591-0d3d416ff680 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:58.825959 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb31573-7cbc-4cd5-9591-0d3d416ff680 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:58.826369 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb31573-7cbc-4cd5-9591-0d3d416ff680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:58.855127 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-ddb31573-7cbc-4cd5-9591-0d3d416ff680 None None] MFA Rules not processed for user `7c2abbae4c894854bbbb227ad5b94444`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:58.856573 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ddb31573-7cbc-4cd5-9591-0d3d416ff680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:58.891098 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d92050-6f6b-4dc8-8bb6-3946aabccfdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:58.891764 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d92050-6f6b-4dc8-8bb6-3946aabccfdf admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:58.892046 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d92050-6f6b-4dc8-8bb6-3946aabccfdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:58.892323 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d92050-6f6b-4dc8-8bb6-3946aabccfdf admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:58.893012 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d92050-6f6b-4dc8-8bb6-3946aabccfdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:58.894934 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d92050-6f6b-4dc8-8bb6-3946aabccfdf admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:58.925937 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1065/2122] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:58 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:58.929584 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d92050-6f6b-4dc8-8bb6-3946aabccfdf admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:58.932001 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1058/2123] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:58.944600 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb31573-7cbc-4cd5-9591-0d3d416ff680 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:58.946688 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ddb31573-7cbc-4cd5-9591-0d3d416ff680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:58.995983 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:59.001080 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d92050-6f6b-4dc8-8bb6-3946aabccfdf admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:59.001825 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:59.009667 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3c215-952e-442b-833e-9c3712b43297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:59.011272 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3c215-952e-442b-833e-9c3712b43297 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:59.012161 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3c215-952e-442b-833e-9c3712b43297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:59.012611 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3c215-952e-442b-833e-9c3712b43297 admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:59.013486 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3c215-952e-442b-833e-9c3712b43297 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:59.015770 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3c215-952e-442b-833e-9c3712b43297 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:59.024704 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb31573-7cbc-4cd5-9591-0d3d416ff680 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:59.040910 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1066/2124] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:58 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:38:59.045348 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3c215-952e-442b-833e-9c3712b43297 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:59.057083 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3c215-952e-442b-833e-9c3712b43297 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:59.062772 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ddb4b4-3e63-4bfd-940b-aadd10564b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:59.063542 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ddb4b4-3e63-4bfd-940b-aadd10564b5d admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:59.063927 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ddb4b4-3e63-4bfd-940b-aadd10564b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:59.064976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ddb4b4-3e63-4bfd-940b-aadd10564b5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:59.065846 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ddb4b4-3e63-4bfd-940b-aadd10564b5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:59.067654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ddb4b4-3e63-4bfd-940b-aadd10564b5d admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:59.079485 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1067/2125] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:59.083060 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ddb4b4-3e63-4bfd-940b-aadd10564b5d admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:59.084533 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e256b4c9-b760-4b09-936c-04df97e6afa8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:59.085337 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256b4c9-b760-4b09-936c-04df97e6afa8 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:59.090627 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256b4c9-b760-4b09-936c-04df97e6afa8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:59.090927 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256b4c9-b760-4b09-936c-04df97e6afa8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:59.095063 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ddb4b4-3e63-4bfd-940b-aadd10564b5d admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:59.100117 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cfbcfa-6eb4-4796-a65e-f993ea28ea72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:59.100902 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfbcfa-6eb4-4796-a65e-f993ea28ea72 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:59.101187 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfbcfa-6eb4-4796-a65e-f993ea28ea72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:59.101464 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfbcfa-6eb4-4796-a65e-f993ea28ea72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:59.102554 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cfbcfa-6eb4-4796-a65e-f993ea28ea72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:59.106462 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cfbcfa-6eb4-4796-a65e-f993ea28ea72 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:59.124357 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1068/2126] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:38:59.125946 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cfbcfa-6eb4-4796-a65e-f993ea28ea72 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:59.141298 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cfbcfa-6eb4-4796-a65e-f993ea28ea72 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:59.152107 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c0f6a3-238d-418f-9f95-6ceb900e247d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:59.152107 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0f6a3-238d-418f-9f95-6ceb900e247d admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:59.152107 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0f6a3-238d-418f-9f95-6ceb900e247d admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:59.152107 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0f6a3-238d-418f-9f95-6ceb900e247d admin admin] PATH_INFO: `/v3/projects/f5965f85f4d6495eb815a40e7e7f9b6a/users/7119ff0019d64f4ab92f26b12b8b53c1/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:59.171909 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e256b4c9-b760-4b09-936c-04df97e6afa8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:59.173371 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e256b4c9-b760-4b09-936c-04df97e6afa8 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:59.175534 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c0f6a3-238d-418f-9f95-6ceb900e247d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5965f85f4d6495eb815a40e7e7f9b6a, user_id=7119ff0019d64f4ab92f26b12b8b53c1, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:59.178443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c0f6a3-238d-418f-9f95-6ceb900e247d admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:59.185389 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1059/2127] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:38:59.196720 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1069/2128] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:59 2022] PUT /identity/v3/projects/f5965f85f4d6495eb815a40e7e7f9b6a/users/7119ff0019d64f4ab92f26b12b8b53c1/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:38:59.207372 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82274efe-3d7a-4ebd-b2c2-af1263d076df None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:59.207663 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82274efe-3d7a-4ebd-b2c2-af1263d076df None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:59.208015 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82274efe-3d7a-4ebd-b2c2-af1263d076df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:59.213844 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c0f6a3-238d-418f-9f95-6ceb900e247d admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:59.238070 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-82274efe-3d7a-4ebd-b2c2-af1263d076df None None] MFA Rules not processed for user `7119ff0019d64f4ab92f26b12b8b53c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:59.239389 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-82274efe-3d7a-4ebd-b2c2-af1263d076df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:59.277568 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c0f6a3-238d-418f-9f95-6ceb900e247d admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:59.282308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb94a0-700a-4f58-8fcf-425ddecdf19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:59.283180 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb94a0-700a-4f58-8fcf-425ddecdf19f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:59.283462 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb94a0-700a-4f58-8fcf-425ddecdf19f admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:59.283737 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb94a0-700a-4f58-8fcf-425ddecdf19f admin admin] PATH_INFO: `/v3/users/7c2abbae4c894854bbbb227ad5b94444` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:59.284957 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bb94a0-700a-4f58-8fcf-425ddecdf19f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c2abbae4c894854bbbb227ad5b94444)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:59.286916 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bb94a0-700a-4f58-8fcf-425ddecdf19f admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:59.305384 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1060/2129] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:59.310466 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82274efe-3d7a-4ebd-b2c2-af1263d076df None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:59.321658 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82274efe-3d7a-4ebd-b2c2-af1263d076df None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:59.326528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6274dadd-ca7e-4684-b62f-4637cb5e895d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:59.327246 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274dadd-ca7e-4684-b62f-4637cb5e895d admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:59.327532 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274dadd-ca7e-4684-b62f-4637cb5e895d admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:59.327805 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274dadd-ca7e-4684-b62f-4637cb5e895d admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:59.328502 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6274dadd-ca7e-4684-b62f-4637cb5e895d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:59.330486 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6274dadd-ca7e-4684-b62f-4637cb5e895d admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:59.346245 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-36bb94a0-700a-4f58-8fcf-425ddecdf19f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c2abbae4c894854bbbb227ad5b94444'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:59.346977 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-36bb94a0-700a-4f58-8fcf-425ddecdf19f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c2abbae4c894854bbbb227ad5b94444'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:59.354889 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-36bb94a0-700a-4f58-8fcf-425ddecdf19f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c2abbae4c894854bbbb227ad5b94444'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:59.355507 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-36bb94a0-700a-4f58-8fcf-425ddecdf19f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c2abbae4c894854bbbb227ad5b94444'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:59.357683 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1061/2130] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:59 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:38:59.361701 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6274dadd-ca7e-4684-b62f-4637cb5e895d admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:59.363227 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-36bb94a0-700a-4f58-8fcf-425ddecdf19f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c2abbae4c894854bbbb227ad5b94444'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:38:59.363347 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6274dadd-ca7e-4684-b62f-4637cb5e895d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:59.380891 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1070/2131] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:38:59 2022] DELETE /identity/v3/users/7c2abbae4c894854bbbb227ad5b94444 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:38:59.382529 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:59.385562 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406850f-a566-4fda-af8a-95bbd2e21088 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:59.385909 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406850f-a566-4fda-af8a-95bbd2e21088 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:59.386454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406850f-a566-4fda-af8a-95bbd2e21088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:59.404016 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-f406850f-a566-4fda-af8a-95bbd2e21088 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:59.405233 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-f406850f-a566-4fda-af8a-95bbd2e21088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:59.421485 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:59.433323 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6274dadd-ca7e-4684-b62f-4637cb5e895d admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:59.473518 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1071/2132] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:59.482937 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f406850f-a566-4fda-af8a-95bbd2e21088 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:59.484452 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-f406850f-a566-4fda-af8a-95bbd2e21088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:59.489865 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef6e0d-4e33-48c1-827c-3329cf09edfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:59.490575 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef6e0d-4e33-48c1-827c-3329cf09edfd admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:59.490867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef6e0d-4e33-48c1-827c-3329cf09edfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:59.491145 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef6e0d-4e33-48c1-827c-3329cf09edfd admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:59.491843 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ef6e0d-4e33-48c1-827c-3329cf09edfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:59.493773 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ef6e0d-4e33-48c1-827c-3329cf09edfd admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:59.514368 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1062/2133] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:59 2022] POST /identity/v3/users => generated 289 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:59.518289 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef6e0d-4e33-48c1-827c-3329cf09edfd admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:59.528611 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef6e0d-4e33-48c1-827c-3329cf09edfd admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:59.533417 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be059736-4553-4b91-b413-76f9c4e30313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:59.534210 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be059736-4553-4b91-b413-76f9c4e30313 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:59.534492 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be059736-4553-4b91-b413-76f9c4e30313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:59.534916 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be059736-4553-4b91-b413-76f9c4e30313 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:59.535647 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be059736-4553-4b91-b413-76f9c4e30313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:59.537238 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be059736-4553-4b91-b413-76f9c4e30313 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:59.545050 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f406850f-a566-4fda-af8a-95bbd2e21088 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:59.547928 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1063/2134] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:38:59.548883 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bda25a-99fc-45a7-9668-b6c67b20065a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:59.549535 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bda25a-99fc-45a7-9668-b6c67b20065a service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:59.549934 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bda25a-99fc-45a7-9668-b6c67b20065a service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:59.550256 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bda25a-99fc-45a7-9668-b6c67b20065a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:59.551643 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be059736-4553-4b91-b413-76f9c4e30313 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:59.560291 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bda25a-99fc-45a7-9668-b6c67b20065a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:59.561237 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be059736-4553-4b91-b413-76f9c4e30313 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:59.561763 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bda25a-99fc-45a7-9668-b6c67b20065a service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:59.566237 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6d6102-aea1-4fb8-bc37-b0795bcf7629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:59.567415 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d6102-aea1-4fb8-bc37-b0795bcf7629 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:59.567766 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d6102-aea1-4fb8-bc37-b0795bcf7629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:59.568047 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d6102-aea1-4fb8-bc37-b0795bcf7629 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:59.568773 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6d6102-aea1-4fb8-bc37-b0795bcf7629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:59.570401 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6d6102-aea1-4fb8-bc37-b0795bcf7629 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:59.571940 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1072/2135] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:38:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:38:59.584452 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1064/2136] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:38:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:38:59.585539 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bda25a-99fc-45a7-9668-b6c67b20065a service neutron] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:59.595475 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bda25a-99fc-45a7-9668-b6c67b20065a service neutron] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:59.599998 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c1f12-db93-4150-870a-5df6b8f20237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:59.600863 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c1f12-db93-4150-870a-5df6b8f20237 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:59.601140 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c1f12-db93-4150-870a-5df6b8f20237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:59.601412 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c1f12-db93-4150-870a-5df6b8f20237 admin admin] PATH_INFO: `/v3/projects/ee879bc90fb44eaab20f022f2b717a70/users/36e485b101c44917a2106ea24af68195/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:59.617460 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c1f12-db93-4150-870a-5df6b8f20237 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee879bc90fb44eaab20f022f2b717a70, user_id=36e485b101c44917a2106ea24af68195, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:59.620199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c1f12-db93-4150-870a-5df6b8f20237 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:59.633167 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1073/2137] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:38:59 2022] PUT /identity/v3/projects/ee879bc90fb44eaab20f022f2b717a70/users/36e485b101c44917a2106ea24af68195/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:38:59.645746 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47636420-8e09-43bb-a6c5-0c03514b309e None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:59.646051 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47636420-8e09-43bb-a6c5-0c03514b309e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:59.646349 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47636420-8e09-43bb-a6c5-0c03514b309e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:59.676263 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-47636420-8e09-43bb-a6c5-0c03514b309e None None] MFA Rules not processed for user `36e485b101c44917a2106ea24af68195`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:38:59.677668 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-47636420-8e09-43bb-a6c5-0c03514b309e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:59.746359 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1065/2138] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:38:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:38:59.751185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c1f12-db93-4150-870a-5df6b8f20237 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:59.831387 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c1f12-db93-4150-870a-5df6b8f20237 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:59.836670 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540dc9c2-93d4-4ead-ad47-bcea64f3d9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:59.837268 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540dc9c2-93d4-4ead-ad47-bcea64f3d9a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:59.837554 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540dc9c2-93d4-4ead-ad47-bcea64f3d9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:59.837836 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540dc9c2-93d4-4ead-ad47-bcea64f3d9a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:59.839169 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540dc9c2-93d4-4ead-ad47-bcea64f3d9a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:59.841498 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540dc9c2-93d4-4ead-ad47-bcea64f3d9a2 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:59.888890 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1074/2139] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:38:59 2022] POST /identity/v3/projects => generated 375 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:38:59.889192 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47636420-8e09-43bb-a6c5-0c03514b309e None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:59.891024 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540dc9c2-93d4-4ead-ad47-bcea64f3d9a2 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:38:59.893258 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-540dc9c2-93d4-4ead-ad47-bcea64f3d9a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:38:59.908417 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47636420-8e09-43bb-a6c5-0c03514b309e None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:59.930446 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:59.930863 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:38:59.976945 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540dc9c2-93d4-4ead-ad47-bcea64f3d9a2 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:38:59.981912 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55b64b9-6224-49b0-9e09-11e199f272e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:59.982896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b64b9-6224-49b0-9e09-11e199f272e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:59.983253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b64b9-6224-49b0-9e09-11e199f272e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:59.983601 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b64b9-6224-49b0-9e09-11e199f272e5 admin admin] PATH_INFO: `/v3/projects/87674e7188fd4e7e9b2bd6b90a19ec99` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:59.984075 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d694cbf5-1079-4380-9d14-1a13dc837323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:38:59.985043 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694cbf5-1079-4380-9d14-1a13dc837323 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:38:59.985330 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694cbf5-1079-4380-9d14-1a13dc837323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:38:59.985601 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694cbf5-1079-4380-9d14-1a13dc837323 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:38:59.986349 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d694cbf5-1079-4380-9d14-1a13dc837323 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:59.987590 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55b64b9-6224-49b0-9e09-11e199f272e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87674e7188fd4e7e9b2bd6b90a19ec99)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:38:59.988277 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d694cbf5-1079-4380-9d14-1a13dc837323 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:38:59.989817 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55b64b9-6224-49b0-9e09-11e199f272e5 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:00.018046 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1066/2140] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:38:59 2022] POST /identity/v3/users => generated 294 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:39:00.022314 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d694cbf5-1079-4380-9d14-1a13dc837323 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:00.033115 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d694cbf5-1079-4380-9d14-1a13dc837323 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:00.039532 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-f55b64b9-6224-49b0-9e09-11e199f272e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87674e7188fd4e7e9b2bd6b90a19ec99', 'request_id': 'req-f55b64b9-6224-49b0-9e09-11e199f272e5'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:00.047602 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-f55b64b9-6224-49b0-9e09-11e199f272e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87674e7188fd4e7e9b2bd6b90a19ec99', 'request_id': 'req-f55b64b9-6224-49b0-9e09-11e199f272e5'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:00.054222 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-f55b64b9-6224-49b0-9e09-11e199f272e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87674e7188fd4e7e9b2bd6b90a19ec99', 'request_id': 'req-f55b64b9-6224-49b0-9e09-11e199f272e5'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:00.066812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-f55b64b9-6224-49b0-9e09-11e199f272e5 admin admin] The token cache is being invalidate because project 87674e7188fd4e7e9b2bd6b90a19ec99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:39:00.067111 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-f55b64b9-6224-49b0-9e09-11e199f272e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:00.067784 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-f55b64b9-6224-49b0-9e09-11e199f272e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:00.069401 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1075/2141] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:38:59 2022] DELETE /identity/v3/projects/87674e7188fd4e7e9b2bd6b90a19ec99 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:39:00.074767 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6784a-ff1d-4322-addf-590b8a9a3ab2 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:00.075046 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6784a-ff1d-4322-addf-590b8a9a3ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:00.075331 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6784a-ff1d-4322-addf-590b8a9a3ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:00.092459 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17a292-0a26-46b7-a935-58304df433fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:00.093057 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17a292-0a26-46b7-a935-58304df433fe admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:00.093164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-0ec6784a-ff1d-4322-addf-590b8a9a3ab2 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:39:00.093343 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17a292-0a26-46b7-a935-58304df433fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:00.093619 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17a292-0a26-46b7-a935-58304df433fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:00.094387 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f17a292-0a26-46b7-a935-58304df433fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:00.094490 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-0ec6784a-ff1d-4322-addf-590b8a9a3ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:00.096003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f17a292-0a26-46b7-a935-58304df433fe admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:00.102748 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1076/2142] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:39:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:39:00.108917 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1067/2143] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:39:00.109749 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17a292-0a26-46b7-a935-58304df433fe admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:00.109922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f831126-61ff-4147-8fb2-9137aad789db None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:00.110256 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f831126-61ff-4147-8fb2-9137aad789db None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:00.110585 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f831126-61ff-4147-8fb2-9137aad789db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:00.111483 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1f17a292-0a26-46b7-a935-58304df433fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:00.124410 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17a292-0a26-46b7-a935-58304df433fe admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:00.129389 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711dfb2c-4e62-4f1c-a7af-2e00de7f4b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:00.130322 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711dfb2c-4e62-4f1c-a7af-2e00de7f4b2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:00.130609 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711dfb2c-4e62-4f1c-a7af-2e00de7f4b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:00.130884 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711dfb2c-4e62-4f1c-a7af-2e00de7f4b2e admin admin] PATH_INFO: `/v3/projects/c07d45272cf8447a9e0cdab035d7b689/users/d41e3eabed6449b6b7756f6625e4f007/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:00.131012 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-7f831126-61ff-4147-8fb2-9137aad789db None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:39:00.132111 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-7f831126-61ff-4147-8fb2-9137aad789db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:00.144537 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1077/2144] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:39:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:39:00.148084 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711dfb2c-4e62-4f1c-a7af-2e00de7f4b2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c07d45272cf8447a9e0cdab035d7b689, user_id=d41e3eabed6449b6b7756f6625e4f007, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:00.148446 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f831126-61ff-4147-8fb2-9137aad789db None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:00.150977 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711dfb2c-4e62-4f1c-a7af-2e00de7f4b2e admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:00.160492 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f831126-61ff-4147-8fb2-9137aad789db None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:00.172207 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1068/2145] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:39:00 2022] PUT /identity/v3/projects/c07d45272cf8447a9e0cdab035d7b689/users/d41e3eabed6449b6b7756f6625e4f007/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:39:00.176443 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711dfb2c-4e62-4f1c-a7af-2e00de7f4b2e admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:00.233174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bcca54-8753-4474-bfcb-7929e1c50631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:00.233837 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bcca54-8753-4474-bfcb-7929e1c50631 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:00.234148 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bcca54-8753-4474-bfcb-7929e1c50631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:00.234431 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bcca54-8753-4474-bfcb-7929e1c50631 admin admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:00.235160 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bcca54-8753-4474-bfcb-7929e1c50631 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:39:00.235468 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bcca54-8753-4474-bfcb-7929e1c50631 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:00.237076 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bcca54-8753-4474-bfcb-7929e1c50631 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:00.246238 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711dfb2c-4e62-4f1c-a7af-2e00de7f4b2e admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:00.252926 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1078/2146] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:39:00.253075 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e0cab7-4622-4d43-a1b6-a17f5310e535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:00.253075 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0cab7-4622-4d43-a1b6-a17f5310e535 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:00.253075 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0cab7-4622-4d43-a1b6-a17f5310e535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:00.253075 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0cab7-4622-4d43-a1b6-a17f5310e535 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:00.253410 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e0cab7-4622-4d43-a1b6-a17f5310e535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:00.253949 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bcca54-8753-4474-bfcb-7929e1c50631 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:00.254804 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e0cab7-4622-4d43-a1b6-a17f5310e535 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:00.264117 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bcca54-8753-4474-bfcb-7929e1c50631 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:00.268316 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1069/2147] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:39:00.269179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5282ac88-f958-4671-a119-3102f2a6f353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:00.269345 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e0cab7-4622-4d43-a1b6-a17f5310e535 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:00.269810 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282ac88-f958-4671-a119-3102f2a6f353 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:00.270222 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282ac88-f958-4671-a119-3102f2a6f353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:00.270503 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282ac88-f958-4671-a119-3102f2a6f353 admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:00.271203 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5282ac88-f958-4671-a119-3102f2a6f353 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:00.273075 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5282ac88-f958-4671-a119-3102f2a6f353 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:00.278638 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e0cab7-4622-4d43-a1b6-a17f5310e535 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:00.283276 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef80f1-f0d7-4560-91b0-b28a366f6eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:00.284175 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef80f1-f0d7-4560-91b0-b28a366f6eb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:00.284519 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef80f1-f0d7-4560-91b0-b28a366f6eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:00.284756 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef80f1-f0d7-4560-91b0-b28a366f6eb3 admin admin] PATH_INFO: `/v3/projects/c07d45272cf8447a9e0cdab035d7b689/users/d41e3eabed6449b6b7756f6625e4f007/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:00.286739 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcef80f1-f0d7-4560-91b0-b28a366f6eb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c07d45272cf8447a9e0cdab035d7b689, user_id=d41e3eabed6449b6b7756f6625e4f007, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:00.289327 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcef80f1-f0d7-4560-91b0-b28a366f6eb3 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:00.298869 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1079/2148] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:39:00 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:39:00.299772 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1070/2149] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:39:00 2022] PUT /identity/v3/projects/c07d45272cf8447a9e0cdab035d7b689/users/d41e3eabed6449b6b7756f6625e4f007/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:39:00.303050 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef80f1-f0d7-4560-91b0-b28a366f6eb3 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:00.309603 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:39:00.309930 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73ae6e-6c22-49b5-aaba-1153fcab055f None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:00.310249 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73ae6e-6c22-49b5-aaba-1153fcab055f None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:00.310527 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73ae6e-6c22-49b5-aaba-1153fcab055f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:00.336933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:39:00.339772 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-1f73ae6e-6c22-49b5-aaba-1153fcab055f None None] MFA Rules not processed for user `d41e3eabed6449b6b7756f6625e4f007`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:39:00.342253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-1f73ae6e-6c22-49b5-aaba-1153fcab055f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:00.372090 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef80f1-f0d7-4560-91b0-b28a366f6eb3 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:00.416061 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1080/2150] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:39:00 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:39:00.427514 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73ae6e-6c22-49b5-aaba-1153fcab055f None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:00.428912 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-1f73ae6e-6c22-49b5-aaba-1153fcab055f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:00.429544 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3120d0-fb1d-4cfd-8136-ae35886a5f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:00.430248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3120d0-fb1d-4cfd-8136-ae35886a5f63 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:00.430533 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3120d0-fb1d-4cfd-8136-ae35886a5f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:00.430808 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3120d0-fb1d-4cfd-8136-ae35886a5f63 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:00.431635 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3120d0-fb1d-4cfd-8136-ae35886a5f63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:00.433605 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3120d0-fb1d-4cfd-8136-ae35886a5f63 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:00.456389 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1071/2151] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:39:00 2022] POST /identity/v3/users => generated 288 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:39:00.462900 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3120d0-fb1d-4cfd-8136-ae35886a5f63 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:00.478002 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3120d0-fb1d-4cfd-8136-ae35886a5f63 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:00.483066 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77b8c9-8a3f-4071-a48b-d501a840102f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:00.483787 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77b8c9-8a3f-4071-a48b-d501a840102f admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:00.484048 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77b8c9-8a3f-4071-a48b-d501a840102f admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:00.484334 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77b8c9-8a3f-4071-a48b-d501a840102f admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:00.485075 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e77b8c9-8a3f-4071-a48b-d501a840102f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:00.487033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e77b8c9-8a3f-4071-a48b-d501a840102f admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:00.494930 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73ae6e-6c22-49b5-aaba-1153fcab055f None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:00.498113 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1072/2152] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:39:00.498504 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c5b85b-943a-4e59-92fb-2d8e31ff8035 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:00.501944 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c5b85b-943a-4e59-92fb-2d8e31ff8035 service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:00.502061 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77b8c9-8a3f-4071-a48b-d501a840102f admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:00.502261 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c5b85b-943a-4e59-92fb-2d8e31ff8035 service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:00.502543 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c5b85b-943a-4e59-92fb-2d8e31ff8035 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:00.504357 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-08c5b85b-943a-4e59-92fb-2d8e31ff8035 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:00.513131 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77b8c9-8a3f-4071-a48b-d501a840102f admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:00.518759 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0b4fc-4e37-4407-a09e-f4e561dae6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:00.519426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0b4fc-4e37-4407-a09e-f4e561dae6d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:00.519748 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0b4fc-4e37-4407-a09e-f4e561dae6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:00.520080 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0b4fc-4e37-4407-a09e-f4e561dae6d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:00.520869 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c0b4fc-4e37-4407-a09e-f4e561dae6d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:00.522625 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c0b4fc-4e37-4407-a09e-f4e561dae6d9 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:00.534350 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1073/2153] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:39:00.538486 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0b4fc-4e37-4407-a09e-f4e561dae6d9 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:00.548242 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0b4fc-4e37-4407-a09e-f4e561dae6d9 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:00.553070 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde313d0-f9f3-44d3-9396-92d25c329b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:00.554064 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde313d0-f9f3-44d3-9396-92d25c329b8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:00.554386 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde313d0-f9f3-44d3-9396-92d25c329b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:00.554659 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde313d0-f9f3-44d3-9396-92d25c329b8d admin admin] PATH_INFO: `/v3/projects/c7aa3271254847f8940db756735695c9/users/22553f097b5a48ccbf0211609f5207ef/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:00.571150 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde313d0-f9f3-44d3-9396-92d25c329b8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7aa3271254847f8940db756735695c9, user_id=22553f097b5a48ccbf0211609f5207ef, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:00.574788 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c5b85b-943a-4e59-92fb-2d8e31ff8035 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:00.574968 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde313d0-f9f3-44d3-9396-92d25c329b8d admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:00.576478 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c5b85b-943a-4e59-92fb-2d8e31ff8035 service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:00.586675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1081/2154] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:39:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:39:00.588175 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1074/2155] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:39:00 2022] PUT /identity/v3/projects/c7aa3271254847f8940db756735695c9/users/22553f097b5a48ccbf0211609f5207ef/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:39:00.598957 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390eedf6-a93b-4051-a1e8-68d490e93f8d None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:00.599217 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390eedf6-a93b-4051-a1e8-68d490e93f8d None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:00.599502 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390eedf6-a93b-4051-a1e8-68d490e93f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:00.628149 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-390eedf6-a93b-4051-a1e8-68d490e93f8d None None] MFA Rules not processed for user `22553f097b5a48ccbf0211609f5207ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:39:00.629293 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-390eedf6-a93b-4051-a1e8-68d490e93f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:00.688129 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:39:00.727418 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1082/2156] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:39:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:39:00.739747 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde313d0-f9f3-44d3-9396-92d25c329b8d admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:00.741488 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-cde313d0-f9f3-44d3-9396-92d25c329b8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:00.817699 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde313d0-f9f3-44d3-9396-92d25c329b8d admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:00.821742 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad0781-a342-498f-a83c-828c1a436cff service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfca4d302aa43249609513f35d55d7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:00.822451 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad0781-a342-498f-a83c-828c1a436cff service glance] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:00.822777 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad0781-a342-498f-a83c-828c1a436cff service glance] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:00.823612 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad0781-a342-498f-a83c-828c1a436cff service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:00.832397 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad0781-a342-498f-a83c-828c1a436cff service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:00.834001 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad0781-a342-498f-a83c-828c1a436cff service glance] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:00.837830 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:39:00.847912 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1075/2157] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:39:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:39:02.138982 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390eedf6-a93b-4051-a1e8-68d490e93f8d None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:02.140366 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-390eedf6-a93b-4051-a1e8-68d490e93f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:02.201487 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390eedf6-a93b-4051-a1e8-68d490e93f8d None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:02.204992 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebfe646-d43d-4bd4-b4f9-b4ef41d242e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:02.205637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebfe646-d43d-4bd4-b4f9-b4ef41d242e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:02.205921 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebfe646-d43d-4bd4-b4f9-b4ef41d242e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:02.206232 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebfe646-d43d-4bd4-b4f9-b4ef41d242e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:02.208015 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-7ebfe646-d43d-4bd4-b4f9-b4ef41d242e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:02.276522 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebfe646-d43d-4bd4-b4f9-b4ef41d242e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:02.277987 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebfe646-d43d-4bd4-b4f9-b4ef41d242e8 service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:02.287880 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1083/2158] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:39:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:39:03.291753 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad0781-a342-498f-a83c-828c1a436cff service glance] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:03.301933 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad0781-a342-498f-a83c-828c1a436cff service glance] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:03.305556 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512da3a1-5f18-4d0f-91fa-c97362ae870f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:03.306239 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512da3a1-5f18-4d0f-91fa-c97362ae870f service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:03.306528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512da3a1-5f18-4d0f-91fa-c97362ae870f service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:03.306808 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512da3a1-5f18-4d0f-91fa-c97362ae870f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:03.382579 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512da3a1-5f18-4d0f-91fa-c97362ae870f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:03.384032 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512da3a1-5f18-4d0f-91fa-c97362ae870f service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:03.394455 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1076/2159] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:39:03 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:39:08.360274 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebfe646-d43d-4bd4-b4f9-b4ef41d242e8 service cinder] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:08.430756 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebfe646-d43d-4bd4-b4f9-b4ef41d242e8 service cinder] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:08.435903 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d415c881-117d-43fb-9a17-02e678bf880c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:08.436781 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d415c881-117d-43fb-9a17-02e678bf880c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:08.437066 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d415c881-117d-43fb-9a17-02e678bf880c admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:08.437345 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d415c881-117d-43fb-9a17-02e678bf880c admin admin] PATH_INFO: `/v3/users/7119ff0019d64f4ab92f26b12b8b53c1` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:08.438801 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d415c881-117d-43fb-9a17-02e678bf880c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7119ff0019d64f4ab92f26b12b8b53c1)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:08.440978 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d415c881-117d-43fb-9a17-02e678bf880c admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:08.484833 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:39:08.491814 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-d415c881-117d-43fb-9a17-02e678bf880c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7119ff0019d64f4ab92f26b12b8b53c1'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:08.492474 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-d415c881-117d-43fb-9a17-02e678bf880c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7119ff0019d64f4ab92f26b12b8b53c1'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:08.499767 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-d415c881-117d-43fb-9a17-02e678bf880c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7119ff0019d64f4ab92f26b12b8b53c1'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:08.500817 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-d415c881-117d-43fb-9a17-02e678bf880c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7119ff0019d64f4ab92f26b12b8b53c1'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:08.508033 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-d415c881-117d-43fb-9a17-02e678bf880c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7119ff0019d64f4ab92f26b12b8b53c1'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:08.525041 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1084/2160] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:39:08 2022] DELETE /identity/v3/users/7119ff0019d64f4ab92f26b12b8b53c1 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:39:08.530378 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1648b40-2277-4452-b2bd-3e88dc8e9d4b None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:08.530889 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1648b40-2277-4452-b2bd-3e88dc8e9d4b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:08.531007 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1648b40-2277-4452-b2bd-3e88dc8e9d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:08.548388 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-e1648b40-2277-4452-b2bd-3e88dc8e9d4b None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:39:08.550145 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e1648b40-2277-4452-b2bd-3e88dc8e9d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:08.613483 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1077/2161] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:39:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:39:08.618733 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d415c881-117d-43fb-9a17-02e678bf880c admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:08.620733 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-d415c881-117d-43fb-9a17-02e678bf880c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:08.679140 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d415c881-117d-43fb-9a17-02e678bf880c admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:08.683391 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2d27f-c3d8-4de9-9f8b-c6d88ffb65a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:08.684037 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2d27f-c3d8-4de9-9f8b-c6d88ffb65a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:08.684350 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2d27f-c3d8-4de9-9f8b-c6d88ffb65a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:08.684734 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2d27f-c3d8-4de9-9f8b-c6d88ffb65a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:08.695634 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f2d27f-c3d8-4de9-9f8b-c6d88ffb65a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:08.697160 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f2d27f-c3d8-4de9-9f8b-c6d88ffb65a9 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:08.707814 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1085/2162] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:39:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:39:08.979578 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1648b40-2277-4452-b2bd-3e88dc8e9d4b None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:08.980992 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e1648b40-2277-4452-b2bd-3e88dc8e9d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:08.993656 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1648b40-2277-4452-b2bd-3e88dc8e9d4b None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:09.000321 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbca120-61e0-4e7a-9ae1-60fb6d7693c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:09.001204 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbca120-61e0-4e7a-9ae1-60fb6d7693c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:09.001486 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbca120-61e0-4e7a-9ae1-60fb6d7693c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:09.001756 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbca120-61e0-4e7a-9ae1-60fb6d7693c9 admin admin] PATH_INFO: `/v3/projects/f5965f85f4d6495eb815a40e7e7f9b6a` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:09.002993 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbca120-61e0-4e7a-9ae1-60fb6d7693c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5965f85f4d6495eb815a40e7e7f9b6a)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:09.004930 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbca120-61e0-4e7a-9ae1-60fb6d7693c9 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:09.050693 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-dfbca120-61e0-4e7a-9ae1-60fb6d7693c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5965f85f4d6495eb815a40e7e7f9b6a', 'request_id': 'req-dfbca120-61e0-4e7a-9ae1-60fb6d7693c9'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:09.058440 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-dfbca120-61e0-4e7a-9ae1-60fb6d7693c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5965f85f4d6495eb815a40e7e7f9b6a', 'request_id': 'req-dfbca120-61e0-4e7a-9ae1-60fb6d7693c9'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:09.064552 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-dfbca120-61e0-4e7a-9ae1-60fb6d7693c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5965f85f4d6495eb815a40e7e7f9b6a', 'request_id': 'req-dfbca120-61e0-4e7a-9ae1-60fb6d7693c9'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:09.076657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-dfbca120-61e0-4e7a-9ae1-60fb6d7693c9 admin admin] The token cache is being invalidate because project f5965f85f4d6495eb815a40e7e7f9b6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:39:09.076949 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-dfbca120-61e0-4e7a-9ae1-60fb6d7693c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:09.077607 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-dfbca120-61e0-4e7a-9ae1-60fb6d7693c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:09.079736 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1078/2163] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:39:08 2022] DELETE /identity/v3/projects/f5965f85f4d6495eb815a40e7e7f9b6a => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:39:09.084063 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2d27f-c3d8-4de9-9f8b-c6d88ffb65a9 service neutron] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:09.094658 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2d27f-c3d8-4de9-9f8b-c6d88ffb65a9 service neutron] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:09.149409 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf33f5c8-fc24-48aa-9110-4434e47d8c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:09.150324 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33f5c8-fc24-48aa-9110-4434e47d8c00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:09.150646 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33f5c8-fc24-48aa-9110-4434e47d8c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:09.150918 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33f5c8-fc24-48aa-9110-4434e47d8c00 admin admin] PATH_INFO: `/v3/users/36e485b101c44917a2106ea24af68195` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:09.152151 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf33f5c8-fc24-48aa-9110-4434e47d8c00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36e485b101c44917a2106ea24af68195)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:09.154004 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf33f5c8-fc24-48aa-9110-4434e47d8c00 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:09.201080 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-bf33f5c8-fc24-48aa-9110-4434e47d8c00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36e485b101c44917a2106ea24af68195'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:09.201780 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-bf33f5c8-fc24-48aa-9110-4434e47d8c00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36e485b101c44917a2106ea24af68195'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:09.208866 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-bf33f5c8-fc24-48aa-9110-4434e47d8c00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36e485b101c44917a2106ea24af68195'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:09.209501 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-bf33f5c8-fc24-48aa-9110-4434e47d8c00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36e485b101c44917a2106ea24af68195'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:09.215186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:39:09.217848 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-bf33f5c8-fc24-48aa-9110-4434e47d8c00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36e485b101c44917a2106ea24af68195'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:09.237306 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1086/2164] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:39:09 2022] DELETE /identity/v3/users/36e485b101c44917a2106ea24af68195 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:39:09.633512 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbca120-61e0-4e7a-9ae1-60fb6d7693c9 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:09.635609 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-dfbca120-61e0-4e7a-9ae1-60fb6d7693c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:09.659759 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbca120-61e0-4e7a-9ae1-60fb6d7693c9 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:09.674917 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:39:09.723880 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92088b1d-0c28-479c-b43c-aa19111eefd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:09.724768 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92088b1d-0c28-479c-b43c-aa19111eefd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:09.725081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92088b1d-0c28-479c-b43c-aa19111eefd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:09.725332 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92088b1d-0c28-479c-b43c-aa19111eefd8 admin admin] PATH_INFO: `/v3/projects/ee879bc90fb44eaab20f022f2b717a70` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:09.726633 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92088b1d-0c28-479c-b43c-aa19111eefd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee879bc90fb44eaab20f022f2b717a70)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:09.728598 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92088b1d-0c28-479c-b43c-aa19111eefd8 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:09.787106 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-92088b1d-0c28-479c-b43c-aa19111eefd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee879bc90fb44eaab20f022f2b717a70', 'request_id': 'req-92088b1d-0c28-479c-b43c-aa19111eefd8'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:09.794333 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-92088b1d-0c28-479c-b43c-aa19111eefd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee879bc90fb44eaab20f022f2b717a70', 'request_id': 'req-92088b1d-0c28-479c-b43c-aa19111eefd8'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:09.800552 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-92088b1d-0c28-479c-b43c-aa19111eefd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee879bc90fb44eaab20f022f2b717a70', 'request_id': 'req-92088b1d-0c28-479c-b43c-aa19111eefd8'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:09.811897 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-92088b1d-0c28-479c-b43c-aa19111eefd8 admin admin] The token cache is being invalidate because project ee879bc90fb44eaab20f022f2b717a70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:39:09.812191 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-92088b1d-0c28-479c-b43c-aa19111eefd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:09.812848 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-92088b1d-0c28-479c-b43c-aa19111eefd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:09.814430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1079/2165] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:39:09 2022] DELETE /identity/v3/projects/ee879bc90fb44eaab20f022f2b717a70 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:39:09.818539 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf33f5c8-fc24-48aa-9110-4434e47d8c00 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:09.819944 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-bf33f5c8-fc24-48aa-9110-4434e47d8c00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:09.833040 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf33f5c8-fc24-48aa-9110-4434e47d8c00 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:09.889346 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a08f29-d333-4345-bb7d-7878f7790ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:09.890162 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a08f29-d333-4345-bb7d-7878f7790ae4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:09.890449 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a08f29-d333-4345-bb7d-7878f7790ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:09.890726 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a08f29-d333-4345-bb7d-7878f7790ae4 admin admin] PATH_INFO: `/v3/users/d41e3eabed6449b6b7756f6625e4f007` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:09.891909 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a08f29-d333-4345-bb7d-7878f7790ae4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d41e3eabed6449b6b7756f6625e4f007)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:09.893839 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a08f29-d333-4345-bb7d-7878f7790ae4 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:09.939844 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-71a08f29-d333-4345-bb7d-7878f7790ae4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd41e3eabed6449b6b7756f6625e4f007'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:09.940502 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-71a08f29-d333-4345-bb7d-7878f7790ae4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd41e3eabed6449b6b7756f6625e4f007'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:09.948280 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-71a08f29-d333-4345-bb7d-7878f7790ae4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd41e3eabed6449b6b7756f6625e4f007'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:09.948877 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-71a08f29-d333-4345-bb7d-7878f7790ae4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd41e3eabed6449b6b7756f6625e4f007'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:09.955898 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-71a08f29-d333-4345-bb7d-7878f7790ae4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd41e3eabed6449b6b7756f6625e4f007'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:09.972774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1087/2166] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:39:09 2022] DELETE /identity/v3/users/d41e3eabed6449b6b7756f6625e4f007 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:39:10.255157 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92088b1d-0c28-479c-b43c-aa19111eefd8 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:10.256668 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-92088b1d-0c28-479c-b43c-aa19111eefd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:10.269332 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92088b1d-0c28-479c-b43c-aa19111eefd8 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:10.329104 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db523a10-0376-42e6-9bc9-8aab41da497b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:10.329964 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db523a10-0376-42e6-9bc9-8aab41da497b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:10.330319 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db523a10-0376-42e6-9bc9-8aab41da497b admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:10.330593 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db523a10-0376-42e6-9bc9-8aab41da497b admin admin] PATH_INFO: `/v3/projects/c07d45272cf8447a9e0cdab035d7b689` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:10.331787 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db523a10-0376-42e6-9bc9-8aab41da497b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c07d45272cf8447a9e0cdab035d7b689)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:10.333723 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db523a10-0376-42e6-9bc9-8aab41da497b admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:10.379793 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:39:10.381198 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-db523a10-0376-42e6-9bc9-8aab41da497b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c07d45272cf8447a9e0cdab035d7b689', 'request_id': 'req-db523a10-0376-42e6-9bc9-8aab41da497b'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:10.390150 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-db523a10-0376-42e6-9bc9-8aab41da497b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c07d45272cf8447a9e0cdab035d7b689', 'request_id': 'req-db523a10-0376-42e6-9bc9-8aab41da497b'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:10.397377 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-db523a10-0376-42e6-9bc9-8aab41da497b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c07d45272cf8447a9e0cdab035d7b689', 'request_id': 'req-db523a10-0376-42e6-9bc9-8aab41da497b'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:10.407903 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-db523a10-0376-42e6-9bc9-8aab41da497b admin admin] The token cache is being invalidate because project c07d45272cf8447a9e0cdab035d7b689 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:39:10.408193 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-db523a10-0376-42e6-9bc9-8aab41da497b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:10.408848 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-db523a10-0376-42e6-9bc9-8aab41da497b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:10.410439 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1080/2167] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:39:10 2022] DELETE /identity/v3/projects/c07d45272cf8447a9e0cdab035d7b689 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:39:10.417271 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43eb1c7-df6e-49eb-9c58-9c182e84e9a8 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:10.417560 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43eb1c7-df6e-49eb-9c58-9c182e84e9a8 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:10.417839 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43eb1c7-df6e-49eb-9c58-9c182e84e9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:10.435510 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-e43eb1c7-df6e-49eb-9c58-9c182e84e9a8 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:39:10.436630 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-e43eb1c7-df6e-49eb-9c58-9c182e84e9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:10.487832 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:39:10.498566 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1088/2168] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:39:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:39:10.506536 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906fd59-8f7b-4425-bcb8-dd16a9f16c82 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:10.506849 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906fd59-8f7b-4425-bcb8-dd16a9f16c82 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:10.507150 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906fd59-8f7b-4425-bcb8-dd16a9f16c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:10.526108 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-d906fd59-8f7b-4425-bcb8-dd16a9f16c82 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:39:10.527490 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d906fd59-8f7b-4425-bcb8-dd16a9f16c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:10.535614 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1081/2169] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:39:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:39:10.539837 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43eb1c7-df6e-49eb-9c58-9c182e84e9a8 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:10.549979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43eb1c7-df6e-49eb-9c58-9c182e84e9a8 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:10.555525 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b731e30-183a-49b6-a177-f637b4928fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:10.556204 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b731e30-183a-49b6-a177-f637b4928fc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:10.556494 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b731e30-183a-49b6-a177-f637b4928fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:10.557287 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b731e30-183a-49b6-a177-f637b4928fc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:10.558066 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b731e30-183a-49b6-a177-f637b4928fc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:39:10.558404 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b731e30-183a-49b6-a177-f637b4928fc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:10.560062 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b731e30-183a-49b6-a177-f637b4928fc5 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:10.576004 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1089/2170] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:39:10.577234 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906fd59-8f7b-4425-bcb8-dd16a9f16c82 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:10.586912 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906fd59-8f7b-4425-bcb8-dd16a9f16c82 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:10.591731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6bbb4d-0a4d-4e8f-9037-5b5401ddbb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:10.592386 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6bbb4d-0a4d-4e8f-9037-5b5401ddbb9a admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:10.592665 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6bbb4d-0a4d-4e8f-9037-5b5401ddbb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:10.592935 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6bbb4d-0a4d-4e8f-9037-5b5401ddbb9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:10.593621 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6bbb4d-0a4d-4e8f-9037-5b5401ddbb9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:10.595555 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6bbb4d-0a4d-4e8f-9037-5b5401ddbb9a admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:10.621754 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1082/2171] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:39:10 2022] POST /identity/v3/projects => generated 397 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:39:10.626344 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b731e30-183a-49b6-a177-f637b4928fc5 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:10.646056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b731e30-183a-49b6-a177-f637b4928fc5 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:10.723855 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90caa54-54d1-4f97-8dc1-aa10fe538b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:10.724528 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90caa54-54d1-4f97-8dc1-aa10fe538b38 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:10.724811 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90caa54-54d1-4f97-8dc1-aa10fe538b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:10.725088 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90caa54-54d1-4f97-8dc1-aa10fe538b38 admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:10.725773 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90caa54-54d1-4f97-8dc1-aa10fe538b38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:10.727787 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90caa54-54d1-4f97-8dc1-aa10fe538b38 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:10.747361 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1090/2172] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:39:10 2022] POST /identity/v3/users => generated 299 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:39:10.751703 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6bbb4d-0a4d-4e8f-9037-5b5401ddbb9a admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:10.762081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6bbb4d-0a4d-4e8f-9037-5b5401ddbb9a admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:10.766919 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0b5e8-5f55-4999-b7dc-a04c47cdfdc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:10.767605 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0b5e8-5f55-4999-b7dc-a04c47cdfdc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:10.767890 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0b5e8-5f55-4999-b7dc-a04c47cdfdc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:10.768168 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0b5e8-5f55-4999-b7dc-a04c47cdfdc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:10.768934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d0b5e8-5f55-4999-b7dc-a04c47cdfdc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:10.770607 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d0b5e8-5f55-4999-b7dc-a04c47cdfdc2 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:10.782141 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1083/2173] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:39:10.786049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90caa54-54d1-4f97-8dc1-aa10fe538b38 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:10.796795 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90caa54-54d1-4f97-8dc1-aa10fe538b38 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:10.801478 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0193eec1-12a9-4d35-a661-d61eeae91418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:10.802180 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193eec1-12a9-4d35-a661-d61eeae91418 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:10.802462 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193eec1-12a9-4d35-a661-d61eeae91418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:10.802742 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193eec1-12a9-4d35-a661-d61eeae91418 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:10.803482 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0193eec1-12a9-4d35-a661-d61eeae91418 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:10.805060 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0193eec1-12a9-4d35-a661-d61eeae91418 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:10.817864 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1091/2174] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:39:10.819185 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0b5e8-5f55-4999-b7dc-a04c47cdfdc2 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:10.829007 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0b5e8-5f55-4999-b7dc-a04c47cdfdc2 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:10.833642 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8ff488-4b8d-4013-9277-edb5ef8b99f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:10.834543 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ff488-4b8d-4013-9277-edb5ef8b99f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:10.834830 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ff488-4b8d-4013-9277-edb5ef8b99f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:10.835204 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ff488-4b8d-4013-9277-edb5ef8b99f6 admin admin] PATH_INFO: `/v3/projects/7fbd6d5b83cf49e0abadec4e97e68b2c/users/8306a25cf5d54bcb973bb3eb4e4cd6aa/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:10.850923 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8ff488-4b8d-4013-9277-edb5ef8b99f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fbd6d5b83cf49e0abadec4e97e68b2c, user_id=8306a25cf5d54bcb973bb3eb4e4cd6aa, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:10.853638 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8ff488-4b8d-4013-9277-edb5ef8b99f6 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:10.866022 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1084/2175] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:39:10 2022] PUT /identity/v3/projects/7fbd6d5b83cf49e0abadec4e97e68b2c/users/8306a25cf5d54bcb973bb3eb4e4cd6aa/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:39:10.878011 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa5e95-bec4-410d-8744-5bbae3929273 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:10.878533 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa5e95-bec4-410d-8744-5bbae3929273 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:10.878624 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa5e95-bec4-410d-8744-5bbae3929273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:10.904843 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-68fa5e95-bec4-410d-8744-5bbae3929273 None None] MFA Rules not processed for user `8306a25cf5d54bcb973bb3eb4e4cd6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:39:10.906051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-68fa5e95-bec4-410d-8744-5bbae3929273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:10.970856 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1092/2176] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:39:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:39:10.982443 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8ff488-4b8d-4013-9277-edb5ef8b99f6 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:10.984120 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0c8ff488-4b8d-4013-9277-edb5ef8b99f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:11.042177 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8ff488-4b8d-4013-9277-edb5ef8b99f6 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:11.045921 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c686ef98-456c-4c96-ae30-1d48a93207b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:11.046596 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686ef98-456c-4c96-ae30-1d48a93207b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:11.046884 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686ef98-456c-4c96-ae30-1d48a93207b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:11.047163 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686ef98-456c-4c96-ae30-1d48a93207b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:11.055503 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c686ef98-456c-4c96-ae30-1d48a93207b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:11.057023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c686ef98-456c-4c96-ae30-1d48a93207b1 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:11.065797 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1085/2177] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:39:10 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:39:20.169766 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fa5e95-bec4-410d-8744-5bbae3929273 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:20.221503 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:39:20.233626 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fa5e95-bec4-410d-8744-5bbae3929273 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:20.238522 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53440af-18c2-4805-8af4-a298cbdd7c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:20.239399 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53440af-18c2-4805-8af4-a298cbdd7c30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:20.239683 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53440af-18c2-4805-8af4-a298cbdd7c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:20.239957 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53440af-18c2-4805-8af4-a298cbdd7c30 admin admin] PATH_INFO: `/v3/users/8306a25cf5d54bcb973bb3eb4e4cd6aa` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:20.241154 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53440af-18c2-4805-8af4-a298cbdd7c30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8306a25cf5d54bcb973bb3eb4e4cd6aa)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:20.243099 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53440af-18c2-4805-8af4-a298cbdd7c30 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:20.292305 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-c53440af-18c2-4805-8af4-a298cbdd7c30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8306a25cf5d54bcb973bb3eb4e4cd6aa'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:20.292944 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-c53440af-18c2-4805-8af4-a298cbdd7c30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8306a25cf5d54bcb973bb3eb4e4cd6aa'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:20.299879 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-c53440af-18c2-4805-8af4-a298cbdd7c30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8306a25cf5d54bcb973bb3eb4e4cd6aa'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:20.300518 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-c53440af-18c2-4805-8af4-a298cbdd7c30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8306a25cf5d54bcb973bb3eb4e4cd6aa'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:20.307967 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-c53440af-18c2-4805-8af4-a298cbdd7c30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8306a25cf5d54bcb973bb3eb4e4cd6aa'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:20.326233 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1093/2178] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:39:20 2022] DELETE /identity/v3/users/8306a25cf5d54bcb973bb3eb4e4cd6aa => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:39:20.330871 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c0c9f-f5e8-4acb-8509-ea1ad796491a None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:20.331361 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c0c9f-f5e8-4acb-8509-ea1ad796491a None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:20.331473 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c0c9f-f5e8-4acb-8509-ea1ad796491a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:20.350364 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-e95c0c9f-f5e8-4acb-8509-ea1ad796491a None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:39:20.351750 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e95c0c9f-f5e8-4acb-8509-ea1ad796491a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:20.415637 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1086/2179] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:39:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:39:20.425138 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53440af-18c2-4805-8af4-a298cbdd7c30 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:20.426666 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-c53440af-18c2-4805-8af4-a298cbdd7c30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:20.485452 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53440af-18c2-4805-8af4-a298cbdd7c30 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:20.489366 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aaf850-37e6-451f-8846-33638a08736b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:20.489999 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aaf850-37e6-451f-8846-33638a08736b service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:20.490310 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aaf850-37e6-451f-8846-33638a08736b service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:20.490582 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aaf850-37e6-451f-8846-33638a08736b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:20.500061 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aaf850-37e6-451f-8846-33638a08736b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:20.501482 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aaf850-37e6-451f-8846-33638a08736b service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:20.512017 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1094/2180] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:39:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:39:20.790276 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c0c9f-f5e8-4acb-8509-ea1ad796491a None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:20.792267 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e95c0c9f-f5e8-4acb-8509-ea1ad796491a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:20.807827 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c0c9f-f5e8-4acb-8509-ea1ad796491a None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:20.812440 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1c7b0a-d620-4ec8-9be7-aa7db69fd31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:20.813310 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c7b0a-d620-4ec8-9be7-aa7db69fd31a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:20.813590 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c7b0a-d620-4ec8-9be7-aa7db69fd31a admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:20.813867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c7b0a-d620-4ec8-9be7-aa7db69fd31a admin admin] PATH_INFO: `/v3/projects/7fbd6d5b83cf49e0abadec4e97e68b2c` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:20.815098 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1c7b0a-d620-4ec8-9be7-aa7db69fd31a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fbd6d5b83cf49e0abadec4e97e68b2c)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:20.817017 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1c7b0a-d620-4ec8-9be7-aa7db69fd31a admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:20.823737 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aaf850-37e6-451f-8846-33638a08736b service neutron] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:20.825550 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-f8aaf850-37e6-451f-8846-33638a08736b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:20.829082 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:39:20.838672 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aaf850-37e6-451f-8846-33638a08736b service neutron] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:20.843414 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9742400-8315-4ed0-a5f2-85c5c323ae83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:20.844345 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9742400-8315-4ed0-a5f2-85c5c323ae83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:20.844628 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9742400-8315-4ed0-a5f2-85c5c323ae83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:20.844906 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9742400-8315-4ed0-a5f2-85c5c323ae83 admin admin] PATH_INFO: `/v3/users/c721d29be82a42b0b54ab7d538857649` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:20.846157 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9742400-8315-4ed0-a5f2-85c5c323ae83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c721d29be82a42b0b54ab7d538857649)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:20.848085 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9742400-8315-4ed0-a5f2-85c5c323ae83 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:20.876498 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-cf1c7b0a-d620-4ec8-9be7-aa7db69fd31a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fbd6d5b83cf49e0abadec4e97e68b2c', 'request_id': 'req-cf1c7b0a-d620-4ec8-9be7-aa7db69fd31a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:20.883871 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-cf1c7b0a-d620-4ec8-9be7-aa7db69fd31a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fbd6d5b83cf49e0abadec4e97e68b2c', 'request_id': 'req-cf1c7b0a-d620-4ec8-9be7-aa7db69fd31a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:20.890818 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-cf1c7b0a-d620-4ec8-9be7-aa7db69fd31a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fbd6d5b83cf49e0abadec4e97e68b2c', 'request_id': 'req-cf1c7b0a-d620-4ec8-9be7-aa7db69fd31a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:20.903234 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-cf1c7b0a-d620-4ec8-9be7-aa7db69fd31a admin admin] The token cache is being invalidate because project 7fbd6d5b83cf49e0abadec4e97e68b2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:39:20.903514 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-cf1c7b0a-d620-4ec8-9be7-aa7db69fd31a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:20.904162 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-cf1c7b0a-d620-4ec8-9be7-aa7db69fd31a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:20.905403 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-b9742400-8315-4ed0-a5f2-85c5c323ae83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c721d29be82a42b0b54ab7d538857649'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:20.905836 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1087/2181] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:39:20 2022] DELETE /identity/v3/projects/7fbd6d5b83cf49e0abadec4e97e68b2c => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:39:20.918205 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-b9742400-8315-4ed0-a5f2-85c5c323ae83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c721d29be82a42b0b54ab7d538857649'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:20.918205 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-b9742400-8315-4ed0-a5f2-85c5c323ae83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c721d29be82a42b0b54ab7d538857649'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:20.918205 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-b9742400-8315-4ed0-a5f2-85c5c323ae83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c721d29be82a42b0b54ab7d538857649'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:20.923676 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-b9742400-8315-4ed0-a5f2-85c5c323ae83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c721d29be82a42b0b54ab7d538857649'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:20.929693 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:39:20.948897 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1095/2182] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:39:20 2022] DELETE /identity/v3/users/c721d29be82a42b0b54ab7d538857649 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:39:20.953652 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb4f7b-74ad-49b4-8a07-5c5a4a0168d4 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:20.954030 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb4f7b-74ad-49b4-8a07-5c5a4a0168d4 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:20.954412 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb4f7b-74ad-49b4-8a07-5c5a4a0168d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:20.973234 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-f3cb4f7b-74ad-49b4-8a07-5c5a4a0168d4 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:39:20.974570 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f3cb4f7b-74ad-49b4-8a07-5c5a4a0168d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:21.041169 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1088/2183] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:39:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:39:21.052089 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9742400-8315-4ed0-a5f2-85c5c323ae83 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:21.053636 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-b9742400-8315-4ed0-a5f2-85c5c323ae83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:21.124559 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9742400-8315-4ed0-a5f2-85c5c323ae83 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:21.128587 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e491fe9d-f855-40dd-8398-bd6d429567b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:21.129237 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491fe9d-f855-40dd-8398-bd6d429567b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:21.129518 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491fe9d-f855-40dd-8398-bd6d429567b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:21.129800 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491fe9d-f855-40dd-8398-bd6d429567b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:21.139109 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e491fe9d-f855-40dd-8398-bd6d429567b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:21.140527 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e491fe9d-f855-40dd-8398-bd6d429567b0 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:21.150802 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1096/2184] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:39:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:39:21.475150 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb4f7b-74ad-49b4-8a07-5c5a4a0168d4 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:21.476706 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f3cb4f7b-74ad-49b4-8a07-5c5a4a0168d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:21.489573 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb4f7b-74ad-49b4-8a07-5c5a4a0168d4 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:21.495659 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02710107-5600-4d98-aa00-8f2bce0a886f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:21.496644 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02710107-5600-4d98-aa00-8f2bce0a886f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:21.497005 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02710107-5600-4d98-aa00-8f2bce0a886f admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:21.497357 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02710107-5600-4d98-aa00-8f2bce0a886f admin admin] PATH_INFO: `/v3/projects/f5a88f690c8b425cac8e649702e157e9` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:21.498801 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02710107-5600-4d98-aa00-8f2bce0a886f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5a88f690c8b425cac8e649702e157e9)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:21.501412 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02710107-5600-4d98-aa00-8f2bce0a886f admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:21.552501 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-02710107-5600-4d98-aa00-8f2bce0a886f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5a88f690c8b425cac8e649702e157e9', 'request_id': 'req-02710107-5600-4d98-aa00-8f2bce0a886f'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:21.560417 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-02710107-5600-4d98-aa00-8f2bce0a886f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5a88f690c8b425cac8e649702e157e9', 'request_id': 'req-02710107-5600-4d98-aa00-8f2bce0a886f'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:21.566854 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-02710107-5600-4d98-aa00-8f2bce0a886f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5a88f690c8b425cac8e649702e157e9', 'request_id': 'req-02710107-5600-4d98-aa00-8f2bce0a886f'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:21.577565 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-02710107-5600-4d98-aa00-8f2bce0a886f admin admin] The token cache is being invalidate because project f5a88f690c8b425cac8e649702e157e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:39:21.577852 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-02710107-5600-4d98-aa00-8f2bce0a886f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:21.578498 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-02710107-5600-4d98-aa00-8f2bce0a886f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:21.580036 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1089/2185] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:39:21 2022] DELETE /identity/v3/projects/f5a88f690c8b425cac8e649702e157e9 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:39:21.584905 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf75a8-a6b6-4a6c-828f-53b628c178b2 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:21.585383 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf75a8-a6b6-4a6c-828f-53b628c178b2 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:21.585505 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf75a8-a6b6-4a6c-828f-53b628c178b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:21.604734 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-0ecf75a8-a6b6-4a6c-828f-53b628c178b2 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:39:21.605968 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-0ecf75a8-a6b6-4a6c-828f-53b628c178b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:21.668718 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1097/2186] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:39:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:39:21.677211 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee80592-7794-487b-bc77-5fc82c4508da None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:21.677497 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee80592-7794-487b-bc77-5fc82c4508da None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:21.677781 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee80592-7794-487b-bc77-5fc82c4508da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:21.696212 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-dee80592-7794-487b-bc77-5fc82c4508da None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:39:21.697916 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-dee80592-7794-487b-bc77-5fc82c4508da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:21.705779 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1090/2187] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:39:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:39:21.709907 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf75a8-a6b6-4a6c-828f-53b628c178b2 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:21.720475 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf75a8-a6b6-4a6c-828f-53b628c178b2 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:21.725053 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14859ad9-109c-43fa-8bc9-ac2ecb684e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:21.725692 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14859ad9-109c-43fa-8bc9-ac2ecb684e00 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:21.725979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14859ad9-109c-43fa-8bc9-ac2ecb684e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:21.726289 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14859ad9-109c-43fa-8bc9-ac2ecb684e00 admin admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:21.727022 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14859ad9-109c-43fa-8bc9-ac2ecb684e00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:39:21.727332 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14859ad9-109c-43fa-8bc9-ac2ecb684e00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:21.728965 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14859ad9-109c-43fa-8bc9-ac2ecb684e00 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:21.740903 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1098/2188] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:39:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:39:21.744918 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee80592-7794-487b-bc77-5fc82c4508da None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:21.754553 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee80592-7794-487b-bc77-5fc82c4508da None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:21.759184 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20eb66f-3ddc-4627-8e88-7e6a92361b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:21.760334 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20eb66f-3ddc-4627-8e88-7e6a92361b60 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:21.760609 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20eb66f-3ddc-4627-8e88-7e6a92361b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:21.760881 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20eb66f-3ddc-4627-8e88-7e6a92361b60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:21.761573 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20eb66f-3ddc-4627-8e88-7e6a92361b60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:21.763478 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20eb66f-3ddc-4627-8e88-7e6a92361b60 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:21.776761 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:39:21.789382 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1091/2189] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:39:21 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:39:21.793752 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14859ad9-109c-43fa-8bc9-ac2ecb684e00 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:21.804455 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14859ad9-109c-43fa-8bc9-ac2ecb684e00 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:21.862400 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1439a325-f63c-4877-825a-0669d6c8f647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:21.863075 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439a325-f63c-4877-825a-0669d6c8f647 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:21.863362 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439a325-f63c-4877-825a-0669d6c8f647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:21.863641 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439a325-f63c-4877-825a-0669d6c8f647 admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:21.864346 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1439a325-f63c-4877-825a-0669d6c8f647 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:21.866284 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1439a325-f63c-4877-825a-0669d6c8f647 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:21.888030 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1099/2190] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:39:21 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:39:21.892493 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20eb66f-3ddc-4627-8e88-7e6a92361b60 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:21.905481 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20eb66f-3ddc-4627-8e88-7e6a92361b60 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:21.912332 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b4d91-245c-4263-a73b-18001bd05001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:21.913019 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b4d91-245c-4263-a73b-18001bd05001 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:21.913303 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b4d91-245c-4263-a73b-18001bd05001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:21.913580 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b4d91-245c-4263-a73b-18001bd05001 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:21.914345 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b4d91-245c-4263-a73b-18001bd05001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:21.915963 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b4d91-245c-4263-a73b-18001bd05001 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:21.927835 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1092/2191] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:39:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:39:21.932493 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1439a325-f63c-4877-825a-0669d6c8f647 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:21.942562 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1439a325-f63c-4877-825a-0669d6c8f647 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:21.947863 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d22e17e-4eeb-442c-83f0-f2ace55dedc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:21.948519 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22e17e-4eeb-442c-83f0-f2ace55dedc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:21.948803 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22e17e-4eeb-442c-83f0-f2ace55dedc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:21.949080 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22e17e-4eeb-442c-83f0-f2ace55dedc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:21.949823 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d22e17e-4eeb-442c-83f0-f2ace55dedc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:21.951493 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d22e17e-4eeb-442c-83f0-f2ace55dedc9 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:21.952953 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:39:21.967559 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1100/2192] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:39:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:39:21.968920 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b4d91-245c-4263-a73b-18001bd05001 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:21.979014 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b4d91-245c-4263-a73b-18001bd05001 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:21.984383 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb06322c-b7e3-4748-ab21-2360f3b1ad8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:21.985292 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06322c-b7e3-4748-ab21-2360f3b1ad8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:21.985576 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06322c-b7e3-4748-ab21-2360f3b1ad8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:21.985853 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06322c-b7e3-4748-ab21-2360f3b1ad8a admin admin] PATH_INFO: `/v3/projects/d6ca6e977576491d98c4478d850aa199/users/6aa525e39ce943b5a8f8f9c8c77943e8/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:22.001916 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb06322c-b7e3-4748-ab21-2360f3b1ad8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6ca6e977576491d98c4478d850aa199, user_id=6aa525e39ce943b5a8f8f9c8c77943e8, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:22.004677 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb06322c-b7e3-4748-ab21-2360f3b1ad8a admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:22.017675 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1093/2193] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:39:21 2022] PUT /identity/v3/projects/d6ca6e977576491d98c4478d850aa199/users/6aa525e39ce943b5a8f8f9c8c77943e8/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:39:22.028418 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16510682-21c7-45e8-8278-b75aebb97c0e None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:22.028823 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16510682-21c7-45e8-8278-b75aebb97c0e None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:22.029111 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16510682-21c7-45e8-8278-b75aebb97c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:22.054950 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-16510682-21c7-45e8-8278-b75aebb97c0e None None] MFA Rules not processed for user `6aa525e39ce943b5a8f8f9c8c77943e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:39:22.056670 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-16510682-21c7-45e8-8278-b75aebb97c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:22.120052 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1101/2194] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:39:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:39:22.132538 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb06322c-b7e3-4748-ab21-2360f3b1ad8a admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:22.134968 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-eb06322c-b7e3-4748-ab21-2360f3b1ad8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:22.191805 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb06322c-b7e3-4748-ab21-2360f3b1ad8a admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:22.195439 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84554d46-4a5d-42a7-a5de-95905bb5014b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:22.196072 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84554d46-4a5d-42a7-a5de-95905bb5014b service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:22.196353 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84554d46-4a5d-42a7-a5de-95905bb5014b service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:22.196626 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84554d46-4a5d-42a7-a5de-95905bb5014b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:22.204841 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84554d46-4a5d-42a7-a5de-95905bb5014b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:22.206295 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84554d46-4a5d-42a7-a5de-95905bb5014b service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:22.215444 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1094/2195] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:39:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:39:24.777900 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16510682-21c7-45e8-8278-b75aebb97c0e None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:24.840903 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16510682-21c7-45e8-8278-b75aebb97c0e None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:24.845370 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba6e07-9c5c-4539-ad43-d0c9c5710997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:24.846268 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba6e07-9c5c-4539-ad43-d0c9c5710997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:24.846553 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba6e07-9c5c-4539-ad43-d0c9c5710997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:24.846827 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba6e07-9c5c-4539-ad43-d0c9c5710997 admin admin] PATH_INFO: `/v3/users/6aa525e39ce943b5a8f8f9c8c77943e8` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:24.848002 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba6e07-9c5c-4539-ad43-d0c9c5710997 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6aa525e39ce943b5a8f8f9c8c77943e8)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:24.849915 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba6e07-9c5c-4539-ad43-d0c9c5710997 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:24.896273 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-45ba6e07-9c5c-4539-ad43-d0c9c5710997 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6aa525e39ce943b5a8f8f9c8c77943e8'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:24.896847 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-45ba6e07-9c5c-4539-ad43-d0c9c5710997 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6aa525e39ce943b5a8f8f9c8c77943e8'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:24.903634 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-45ba6e07-9c5c-4539-ad43-d0c9c5710997 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6aa525e39ce943b5a8f8f9c8c77943e8'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:24.904260 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-45ba6e07-9c5c-4539-ad43-d0c9c5710997 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6aa525e39ce943b5a8f8f9c8c77943e8'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:24.911028 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-45ba6e07-9c5c-4539-ad43-d0c9c5710997 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6aa525e39ce943b5a8f8f9c8c77943e8'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:24.927924 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1102/2196] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:39:24 2022] DELETE /identity/v3/users/6aa525e39ce943b5a8f8f9c8c77943e8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:39:24.933893 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2808ad-3dec-4f9a-a6cf-1fdfee786820 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:24.934413 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2808ad-3dec-4f9a-a6cf-1fdfee786820 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:24.934965 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2808ad-3dec-4f9a-a6cf-1fdfee786820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:24.952860 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-6d2808ad-3dec-4f9a-a6cf-1fdfee786820 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:39:24.954061 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6d2808ad-3dec-4f9a-a6cf-1fdfee786820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:25.011264 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1095/2197] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:39:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:39:25.020701 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba6e07-9c5c-4539-ad43-d0c9c5710997 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:25.022131 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-45ba6e07-9c5c-4539-ad43-d0c9c5710997 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:25.028738 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:39:25.087098 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba6e07-9c5c-4539-ad43-d0c9c5710997 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:25.091821 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2873df19-07a9-4d0c-a6f7-1a876c99f4e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:25.092638 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873df19-07a9-4d0c-a6f7-1a876c99f4e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:25.092929 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873df19-07a9-4d0c-a6f7-1a876c99f4e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:25.093207 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873df19-07a9-4d0c-a6f7-1a876c99f4e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:25.102442 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2873df19-07a9-4d0c-a6f7-1a876c99f4e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:25.103917 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2873df19-07a9-4d0c-a6f7-1a876c99f4e7 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:25.113196 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1103/2198] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:39:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:39:25.387508 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2808ad-3dec-4f9a-a6cf-1fdfee786820 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:25.389314 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6d2808ad-3dec-4f9a-a6cf-1fdfee786820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:25.402853 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2808ad-3dec-4f9a-a6cf-1fdfee786820 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:25.413057 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e5a4f6-db87-4e22-9b69-8f910902fb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:25.413972 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e5a4f6-db87-4e22-9b69-8f910902fb56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:25.414290 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e5a4f6-db87-4e22-9b69-8f910902fb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:25.414961 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e5a4f6-db87-4e22-9b69-8f910902fb56 admin admin] PATH_INFO: `/v3/projects/d6ca6e977576491d98c4478d850aa199` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:25.415892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e5a4f6-db87-4e22-9b69-8f910902fb56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6ca6e977576491d98c4478d850aa199)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:25.417707 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e5a4f6-db87-4e22-9b69-8f910902fb56 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:25.466156 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-c7e5a4f6-db87-4e22-9b69-8f910902fb56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6ca6e977576491d98c4478d850aa199', 'request_id': 'req-c7e5a4f6-db87-4e22-9b69-8f910902fb56'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:25.473915 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-c7e5a4f6-db87-4e22-9b69-8f910902fb56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6ca6e977576491d98c4478d850aa199', 'request_id': 'req-c7e5a4f6-db87-4e22-9b69-8f910902fb56'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:25.475273 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:INSERT=3,keystone:SELECT=92,keystone:DELETE=5 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:39:25.481510 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-c7e5a4f6-db87-4e22-9b69-8f910902fb56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6ca6e977576491d98c4478d850aa199', 'request_id': 'req-c7e5a4f6-db87-4e22-9b69-8f910902fb56'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:25.504272 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-c7e5a4f6-db87-4e22-9b69-8f910902fb56 admin admin] The token cache is being invalidate because project d6ca6e977576491d98c4478d850aa199 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:39:25.504691 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-c7e5a4f6-db87-4e22-9b69-8f910902fb56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:25.505436 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-c7e5a4f6-db87-4e22-9b69-8f910902fb56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:25.507231 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1096/2199] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:39:25 2022] DELETE /identity/v3/projects/d6ca6e977576491d98c4478d850aa199 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:39:35.109211 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=24 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:39:35.501945 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=5,keystone:DELETE=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:39:38.383754 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2873df19-07a9-4d0c-a6f7-1a876c99f4e7 service neutron] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:38.385195 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-2873df19-07a9-4d0c-a6f7-1a876c99f4e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:38.397748 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2873df19-07a9-4d0c-a6f7-1a876c99f4e7 service neutron] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:38.453171 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb807b6-8364-4891-a033-ec2ef122b4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:38.454053 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb807b6-8364-4891-a033-ec2ef122b4c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:38.454373 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb807b6-8364-4891-a033-ec2ef122b4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:38.454649 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb807b6-8364-4891-a033-ec2ef122b4c6 admin admin] PATH_INFO: `/v3/users/22553f097b5a48ccbf0211609f5207ef` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:38.455874 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb807b6-8364-4891-a033-ec2ef122b4c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22553f097b5a48ccbf0211609f5207ef)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:38.457816 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb807b6-8364-4891-a033-ec2ef122b4c6 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:38.518268 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-7eb807b6-8364-4891-a033-ec2ef122b4c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22553f097b5a48ccbf0211609f5207ef'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:38.519070 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-7eb807b6-8364-4891-a033-ec2ef122b4c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22553f097b5a48ccbf0211609f5207ef'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:38.527010 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-7eb807b6-8364-4891-a033-ec2ef122b4c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22553f097b5a48ccbf0211609f5207ef'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:38.527630 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-7eb807b6-8364-4891-a033-ec2ef122b4c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22553f097b5a48ccbf0211609f5207ef'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:38.534852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-7eb807b6-8364-4891-a033-ec2ef122b4c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22553f097b5a48ccbf0211609f5207ef'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:38.553916 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1104/2200] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:39:38 2022] DELETE /identity/v3/users/22553f097b5a48ccbf0211609f5207ef => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:39:38.558565 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f60d6-1782-4a18-9f43-e38037502efa None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:38.559050 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f60d6-1782-4a18-9f43-e38037502efa None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:38.559166 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f60d6-1782-4a18-9f43-e38037502efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:38.577697 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-611f60d6-1782-4a18-9f43-e38037502efa None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:39:38.579153 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-611f60d6-1782-4a18-9f43-e38037502efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:38.645201 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1097/2201] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:39:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:39:38.655304 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb807b6-8364-4891-a033-ec2ef122b4c6 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:38.656710 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-7eb807b6-8364-4891-a033-ec2ef122b4c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:38.714430 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb807b6-8364-4891-a033-ec2ef122b4c6 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:38.717949 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be7f05-247a-4f17-abd3-8eb054688b59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:38.718630 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be7f05-247a-4f17-abd3-8eb054688b59 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:38.718912 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be7f05-247a-4f17-abd3-8eb054688b59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:38.719186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be7f05-247a-4f17-abd3-8eb054688b59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:38.728375 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93be7f05-247a-4f17-abd3-8eb054688b59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:38.729789 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93be7f05-247a-4f17-abd3-8eb054688b59 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:38.739189 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1105/2202] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:39:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:39:39.005713 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f60d6-1782-4a18-9f43-e38037502efa None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:39.007195 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-611f60d6-1782-4a18-9f43-e38037502efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:39.022118 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f60d6-1782-4a18-9f43-e38037502efa None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:39.028344 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14db3883-a6b4-467f-beb1-4ce1900727a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:39.029218 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db3883-a6b4-467f-beb1-4ce1900727a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:39.029474 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db3883-a6b4-467f-beb1-4ce1900727a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:39.029864 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db3883-a6b4-467f-beb1-4ce1900727a0 admin admin] PATH_INFO: `/v3/projects/c7aa3271254847f8940db756735695c9` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:39.030976 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14db3883-a6b4-467f-beb1-4ce1900727a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7aa3271254847f8940db756735695c9)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:39.032877 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14db3883-a6b4-467f-beb1-4ce1900727a0 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:39.079430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-14db3883-a6b4-467f-beb1-4ce1900727a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7aa3271254847f8940db756735695c9', 'request_id': 'req-14db3883-a6b4-467f-beb1-4ce1900727a0'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:39.086289 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-14db3883-a6b4-467f-beb1-4ce1900727a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7aa3271254847f8940db756735695c9', 'request_id': 'req-14db3883-a6b4-467f-beb1-4ce1900727a0'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:39.092373 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-14db3883-a6b4-467f-beb1-4ce1900727a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7aa3271254847f8940db756735695c9', 'request_id': 'req-14db3883-a6b4-467f-beb1-4ce1900727a0'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:39.102487 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-14db3883-a6b4-467f-beb1-4ce1900727a0 admin admin] The token cache is being invalidate because project c7aa3271254847f8940db756735695c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:39:39.102770 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-14db3883-a6b4-467f-beb1-4ce1900727a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:39.103392 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-14db3883-a6b4-467f-beb1-4ce1900727a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:39.104907 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1098/2203] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:39:39 2022] DELETE /identity/v3/projects/c7aa3271254847f8940db756735695c9 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:39:39.109766 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf5753-551d-4b87-860a-ae01dc9cc47e None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:39.110064 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf5753-551d-4b87-860a-ae01dc9cc47e None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:39.110384 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf5753-551d-4b87-860a-ae01dc9cc47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:39.129051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-4bcf5753-551d-4b87-860a-ae01dc9cc47e None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:39:39.130276 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-4bcf5753-551d-4b87-860a-ae01dc9cc47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:39.147125 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:39:39.189968 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1106/2204] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:39:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:39:39.198579 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47afac9-d56e-4f64-a71a-30e7ecbdf24f None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:39.198869 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47afac9-d56e-4f64-a71a-30e7ecbdf24f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:39.199146 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47afac9-d56e-4f64-a71a-30e7ecbdf24f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:39.217330 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-f47afac9-d56e-4f64-a71a-30e7ecbdf24f None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:39:39.218548 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f47afac9-d56e-4f64-a71a-30e7ecbdf24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:39.226272 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1099/2205] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:39:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:39:39.230721 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf5753-551d-4b87-860a-ae01dc9cc47e None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:39.240200 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf5753-551d-4b87-860a-ae01dc9cc47e None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:39.244734 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36994dbf-4849-4360-941c-1f10d02a9bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:39.245383 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36994dbf-4849-4360-941c-1f10d02a9bc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:39.245670 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36994dbf-4849-4360-941c-1f10d02a9bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:39.245951 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36994dbf-4849-4360-941c-1f10d02a9bc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:39.246709 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36994dbf-4849-4360-941c-1f10d02a9bc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:39:39.247020 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36994dbf-4849-4360-941c-1f10d02a9bc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:39.248691 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36994dbf-4849-4360-941c-1f10d02a9bc9 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:39.264141 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1107/2206] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:39:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:39:39.265271 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47afac9-d56e-4f64-a71a-30e7ecbdf24f None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:39.274913 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47afac9-d56e-4f64-a71a-30e7ecbdf24f None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:39.279590 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739b5a00-3da5-4d73-8cde-d2c0795257af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:39.280231 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b5a00-3da5-4d73-8cde-d2c0795257af admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:39.280512 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b5a00-3da5-4d73-8cde-d2c0795257af admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:39.280786 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b5a00-3da5-4d73-8cde-d2c0795257af admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:39.281471 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739b5a00-3da5-4d73-8cde-d2c0795257af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:39.283392 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739b5a00-3da5-4d73-8cde-d2c0795257af admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:39.313628 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1100/2207] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:39:39 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:39:39.314926 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36994dbf-4849-4360-941c-1f10d02a9bc9 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:39.325174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36994dbf-4849-4360-941c-1f10d02a9bc9 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:39.385637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e8b57-731f-4933-a58d-225eaec647f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:39.386453 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e8b57-731f-4933-a58d-225eaec647f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:39.386739 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e8b57-731f-4933-a58d-225eaec647f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:39.387017 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e8b57-731f-4933-a58d-225eaec647f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:39.387712 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e8b57-731f-4933-a58d-225eaec647f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:39.389661 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e8b57-731f-4933-a58d-225eaec647f5 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:39.413556 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1108/2208] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:39:39 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:39:39.415070 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739b5a00-3da5-4d73-8cde-d2c0795257af admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:39.425576 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739b5a00-3da5-4d73-8cde-d2c0795257af admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:39.430254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3ad7f-108a-47e9-aac4-00425a542e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:39.430933 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3ad7f-108a-47e9-aac4-00425a542e7a admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:39.431212 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3ad7f-108a-47e9-aac4-00425a542e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:39.431486 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3ad7f-108a-47e9-aac4-00425a542e7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:39.432458 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d3ad7f-108a-47e9-aac4-00425a542e7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:39.435550 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d3ad7f-108a-47e9-aac4-00425a542e7a admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:39.448419 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1101/2209] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:39:39.452525 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e8b57-731f-4933-a58d-225eaec647f5 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:39.463260 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e8b57-731f-4933-a58d-225eaec647f5 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:39.467774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc7d0eb-f921-4901-9a76-e4ae59ee0ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:39.468424 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7d0eb-f921-4901-9a76-e4ae59ee0ee8 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:39.468707 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7d0eb-f921-4901-9a76-e4ae59ee0ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:39.469006 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7d0eb-f921-4901-9a76-e4ae59ee0ee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:39.469731 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc7d0eb-f921-4901-9a76-e4ae59ee0ee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:39.471369 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc7d0eb-f921-4901-9a76-e4ae59ee0ee8 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:39.481112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1109/2210] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:39:39.485074 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3ad7f-108a-47e9-aac4-00425a542e7a admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:39.494991 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3ad7f-108a-47e9-aac4-00425a542e7a admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:39.499975 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a4fb5-4560-44e1-873d-e724ed1d49a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:39.500786 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a4fb5-4560-44e1-873d-e724ed1d49a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:39.501067 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a4fb5-4560-44e1-873d-e724ed1d49a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:39.501345 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a4fb5-4560-44e1-873d-e724ed1d49a9 admin admin] PATH_INFO: `/v3/projects/5843385a09b84387a1fe857084541be4/users/5c1bf39dbb384ed5b951275b1c630508/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:39.516044 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a4fb5-4560-44e1-873d-e724ed1d49a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5843385a09b84387a1fe857084541be4, user_id=5c1bf39dbb384ed5b951275b1c630508, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:39.518733 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a4fb5-4560-44e1-873d-e724ed1d49a9 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:39.531270 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1102/2211] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:39:39 2022] PUT /identity/v3/projects/5843385a09b84387a1fe857084541be4/users/5c1bf39dbb384ed5b951275b1c630508/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:39:39.541335 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22154d05-5285-44f3-a96a-7f52060f4e72 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:39.541625 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22154d05-5285-44f3-a96a-7f52060f4e72 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:39.541902 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22154d05-5285-44f3-a96a-7f52060f4e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:39.567207 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-22154d05-5285-44f3-a96a-7f52060f4e72 None None] MFA Rules not processed for user `5c1bf39dbb384ed5b951275b1c630508`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:39:39.568393 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-22154d05-5285-44f3-a96a-7f52060f4e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:39.631224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1110/2212] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:39:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:39:39.635590 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a4fb5-4560-44e1-873d-e724ed1d49a9 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:39.655910 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:39:39.700307 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a4fb5-4560-44e1-873d-e724ed1d49a9 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:39.705294 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5819ce5-e9d2-4bc6-85c6-93309f91a34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:39.705941 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5819ce5-e9d2-4bc6-85c6-93309f91a34a admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:39.706254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5819ce5-e9d2-4bc6-85c6-93309f91a34a admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:39.706526 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5819ce5-e9d2-4bc6-85c6-93309f91a34a admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:39.707222 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5819ce5-e9d2-4bc6-85c6-93309f91a34a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:39.709139 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5819ce5-e9d2-4bc6-85c6-93309f91a34a admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:39.734792 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1103/2213] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:39:39 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:39:39.738793 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22154d05-5285-44f3-a96a-7f52060f4e72 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:39.750186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22154d05-5285-44f3-a96a-7f52060f4e72 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:39.762002 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:39:39.804929 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f16ac4-bdfd-49b5-a368-319420a5352d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:39.806179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f16ac4-bdfd-49b5-a368-319420a5352d admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:39.806465 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f16ac4-bdfd-49b5-a368-319420a5352d admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:39.806741 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f16ac4-bdfd-49b5-a368-319420a5352d admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:39.807433 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f16ac4-bdfd-49b5-a368-319420a5352d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:39.809351 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f16ac4-bdfd-49b5-a368-319420a5352d admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:39.827255 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1111/2214] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:39:39 2022] POST /identity/v3/users => generated 294 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:39:39.831168 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5819ce5-e9d2-4bc6-85c6-93309f91a34a admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:39.842016 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5819ce5-e9d2-4bc6-85c6-93309f91a34a admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:39.846584 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1216b1-73fc-444a-a6c4-3a057e49002e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:39.847268 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1216b1-73fc-444a-a6c4-3a057e49002e admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:39.847549 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1216b1-73fc-444a-a6c4-3a057e49002e admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:39.847828 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1216b1-73fc-444a-a6c4-3a057e49002e admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:39.848548 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1216b1-73fc-444a-a6c4-3a057e49002e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:39.850162 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1216b1-73fc-444a-a6c4-3a057e49002e admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:39.861927 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1104/2215] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:39:39.865756 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f16ac4-bdfd-49b5-a368-319420a5352d admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:39.875654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f16ac4-bdfd-49b5-a368-319420a5352d admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:39.880213 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549be58e-95f5-42e8-9682-dc56c2c25d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:39.881069 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549be58e-95f5-42e8-9682-dc56c2c25d18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:39.881347 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549be58e-95f5-42e8-9682-dc56c2c25d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:39.881625 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549be58e-95f5-42e8-9682-dc56c2c25d18 admin admin] PATH_INFO: `/v3/projects/a5e0e0eab92e4036a0675d18e0c99391/users/2af31f0f64404b0183fa28c19b33c351/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:39.897716 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549be58e-95f5-42e8-9682-dc56c2c25d18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5e0e0eab92e4036a0675d18e0c99391, user_id=2af31f0f64404b0183fa28c19b33c351, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:39.900912 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549be58e-95f5-42e8-9682-dc56c2c25d18 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:39.913116 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1112/2216] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:39:39 2022] PUT /identity/v3/projects/a5e0e0eab92e4036a0675d18e0c99391/users/2af31f0f64404b0183fa28c19b33c351/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:39:39.917493 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1216b1-73fc-444a-a6c4-3a057e49002e admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:39.927399 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1216b1-73fc-444a-a6c4-3a057e49002e admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:39.982615 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b6ae8-249b-4280-90df-db0cf860917d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:39.983281 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b6ae8-249b-4280-90df-db0cf860917d admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:39.983568 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b6ae8-249b-4280-90df-db0cf860917d admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:39.983846 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b6ae8-249b-4280-90df-db0cf860917d admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:39.984577 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7b6ae8-249b-4280-90df-db0cf860917d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:39.986199 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7b6ae8-249b-4280-90df-db0cf860917d admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:39.995906 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1105/2217] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:39:40.000393 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549be58e-95f5-42e8-9682-dc56c2c25d18 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:40.010783 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549be58e-95f5-42e8-9682-dc56c2c25d18 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:40.015540 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e0baf4-a091-4ce2-881b-4b9642280a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:40.016423 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0baf4-a091-4ce2-881b-4b9642280a81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:40.016700 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0baf4-a091-4ce2-881b-4b9642280a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:40.016977 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0baf4-a091-4ce2-881b-4b9642280a81 admin admin] PATH_INFO: `/v3/projects/a5e0e0eab92e4036a0675d18e0c99391/users/2af31f0f64404b0183fa28c19b33c351/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:40.019016 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e0baf4-a091-4ce2-881b-4b9642280a81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5e0e0eab92e4036a0675d18e0c99391, user_id=2af31f0f64404b0183fa28c19b33c351, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:40.021649 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e0baf4-a091-4ce2-881b-4b9642280a81 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:40.032669 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1113/2218] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:39:39 2022] PUT /identity/v3/projects/a5e0e0eab92e4036a0675d18e0c99391/users/2af31f0f64404b0183fa28c19b33c351/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:39:40.044742 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a493080-fc9b-4e20-a3c9-d41a82359977 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:40.045030 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a493080-fc9b-4e20-a3c9-d41a82359977 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:40.045306 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a493080-fc9b-4e20-a3c9-d41a82359977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:40.072074 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-1a493080-fc9b-4e20-a3c9-d41a82359977 None None] MFA Rules not processed for user `2af31f0f64404b0183fa28c19b33c351`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:39:40.073429 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1a493080-fc9b-4e20-a3c9-d41a82359977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:40.144814 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1106/2219] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:39:40 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:39:40.155886 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e0baf4-a091-4ce2-881b-4b9642280a81 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:40.158068 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-50e0baf4-a091-4ce2-881b-4b9642280a81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:40.217458 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e0baf4-a091-4ce2-881b-4b9642280a81 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:40.221781 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ca30d5-db2b-4c95-94a5-b1b394b72e35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:40.222455 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca30d5-db2b-4c95-94a5-b1b394b72e35 service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:40.222739 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca30d5-db2b-4c95-94a5-b1b394b72e35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:40.223014 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca30d5-db2b-4c95-94a5-b1b394b72e35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:40.281523 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ca30d5-db2b-4c95-94a5-b1b394b72e35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:40.283050 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ca30d5-db2b-4c95-94a5-b1b394b72e35 service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:40.292987 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1114/2220] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:39:40 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:39:43.964851 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a493080-fc9b-4e20-a3c9-d41a82359977 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:43.982334 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:39:44.066587 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a493080-fc9b-4e20-a3c9-d41a82359977 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:44.073393 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc7b71a-40db-4899-b2ed-f61934bf4589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:44.074428 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7b71a-40db-4899-b2ed-f61934bf4589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:44.074780 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7b71a-40db-4899-b2ed-f61934bf4589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:44.075124 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7b71a-40db-4899-b2ed-f61934bf4589 admin admin] PATH_INFO: `/v3/users/5c1bf39dbb384ed5b951275b1c630508` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:44.076597 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc7b71a-40db-4899-b2ed-f61934bf4589 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c1bf39dbb384ed5b951275b1c630508)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:44.078895 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc7b71a-40db-4899-b2ed-f61934bf4589 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:44.155978 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ffc7b71a-40db-4899-b2ed-f61934bf4589 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c1bf39dbb384ed5b951275b1c630508'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:44.157235 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ffc7b71a-40db-4899-b2ed-f61934bf4589 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c1bf39dbb384ed5b951275b1c630508'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:44.157845 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ffc7b71a-40db-4899-b2ed-f61934bf4589 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c1bf39dbb384ed5b951275b1c630508'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:44.174550 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ffc7b71a-40db-4899-b2ed-f61934bf4589 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c1bf39dbb384ed5b951275b1c630508'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:44.182860 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ffc7b71a-40db-4899-b2ed-f61934bf4589 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c1bf39dbb384ed5b951275b1c630508'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:44.207102 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1107/2221] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:39:43 2022] DELETE /identity/v3/users/5c1bf39dbb384ed5b951275b1c630508 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:39:44.211647 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c42b7-9d83-46f4-909d-c283c7caf1b2 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:44.212140 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c42b7-9d83-46f4-909d-c283c7caf1b2 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:44.212242 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c42b7-9d83-46f4-909d-c283c7caf1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:44.230136 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-603c42b7-9d83-46f4-909d-c283c7caf1b2 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:39:44.231313 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-603c42b7-9d83-46f4-909d-c283c7caf1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:44.249750 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:39:44.293585 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1115/2222] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:39:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:39:44.303201 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc7b71a-40db-4899-b2ed-f61934bf4589 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:44.304673 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ffc7b71a-40db-4899-b2ed-f61934bf4589 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:44.365165 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc7b71a-40db-4899-b2ed-f61934bf4589 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:44.368837 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d52bed-d605-4aca-92c0-e03ed4f8c237 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:44.369479 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d52bed-d605-4aca-92c0-e03ed4f8c237 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:44.369761 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d52bed-d605-4aca-92c0-e03ed4f8c237 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:44.370034 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d52bed-d605-4aca-92c0-e03ed4f8c237 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:44.379299 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d52bed-d605-4aca-92c0-e03ed4f8c237 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:44.380731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d52bed-d605-4aca-92c0-e03ed4f8c237 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:44.390320 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1108/2223] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:39:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:39:44.653146 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c42b7-9d83-46f4-909d-c283c7caf1b2 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:44.654671 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-603c42b7-9d83-46f4-909d-c283c7caf1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:44.667416 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c42b7-9d83-46f4-909d-c283c7caf1b2 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:44.677805 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d80968-cbaa-436a-bbd8-336c60aba0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:44.677805 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d80968-cbaa-436a-bbd8-336c60aba0d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:44.677805 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d80968-cbaa-436a-bbd8-336c60aba0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:44.677805 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d80968-cbaa-436a-bbd8-336c60aba0d6 admin admin] PATH_INFO: `/v3/projects/5843385a09b84387a1fe857084541be4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:44.678481 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d80968-cbaa-436a-bbd8-336c60aba0d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5843385a09b84387a1fe857084541be4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:44.678481 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d80968-cbaa-436a-bbd8-336c60aba0d6 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:44.725839 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-72d80968-cbaa-436a-bbd8-336c60aba0d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5843385a09b84387a1fe857084541be4', 'request_id': 'req-72d80968-cbaa-436a-bbd8-336c60aba0d6'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:44.733421 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-72d80968-cbaa-436a-bbd8-336c60aba0d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5843385a09b84387a1fe857084541be4', 'request_id': 'req-72d80968-cbaa-436a-bbd8-336c60aba0d6'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:44.739621 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-72d80968-cbaa-436a-bbd8-336c60aba0d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5843385a09b84387a1fe857084541be4', 'request_id': 'req-72d80968-cbaa-436a-bbd8-336c60aba0d6'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:44.750936 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-72d80968-cbaa-436a-bbd8-336c60aba0d6 admin admin] The token cache is being invalidate because project 5843385a09b84387a1fe857084541be4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:39:44.751226 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-72d80968-cbaa-436a-bbd8-336c60aba0d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:44.751880 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-72d80968-cbaa-436a-bbd8-336c60aba0d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:44.753435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1116/2224] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:39:44 2022] DELETE /identity/v3/projects/5843385a09b84387a1fe857084541be4 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:39:44.757311 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d52bed-d605-4aca-92c0-e03ed4f8c237 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:44.766856 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d52bed-d605-4aca-92c0-e03ed4f8c237 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:44.803362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:39:44.828186 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1703bdd-4d09-4f8b-90e2-f9015e481294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:44.829057 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1703bdd-4d09-4f8b-90e2-f9015e481294 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:44.829340 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1703bdd-4d09-4f8b-90e2-f9015e481294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:44.829613 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1703bdd-4d09-4f8b-90e2-f9015e481294 admin admin] PATH_INFO: `/v3/users/2af31f0f64404b0183fa28c19b33c351` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:44.830821 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1703bdd-4d09-4f8b-90e2-f9015e481294 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2af31f0f64404b0183fa28c19b33c351)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:44.832739 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1703bdd-4d09-4f8b-90e2-f9015e481294 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:44.877155 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-d1703bdd-4d09-4f8b-90e2-f9015e481294 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2af31f0f64404b0183fa28c19b33c351'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:44.877793 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-d1703bdd-4d09-4f8b-90e2-f9015e481294 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2af31f0f64404b0183fa28c19b33c351'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:44.878418 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-d1703bdd-4d09-4f8b-90e2-f9015e481294 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2af31f0f64404b0183fa28c19b33c351'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:44.889999 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-d1703bdd-4d09-4f8b-90e2-f9015e481294 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2af31f0f64404b0183fa28c19b33c351'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:44.896714 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-d1703bdd-4d09-4f8b-90e2-f9015e481294 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2af31f0f64404b0183fa28c19b33c351'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:44.909294 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1109/2225] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:39:44 2022] DELETE /identity/v3/users/2af31f0f64404b0183fa28c19b33c351 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:39:45.184049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d80968-cbaa-436a-bbd8-336c60aba0d6 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:45.185856 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-72d80968-cbaa-436a-bbd8-336c60aba0d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:45.200821 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d80968-cbaa-436a-bbd8-336c60aba0d6 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:45.270011 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ec6f3-5f78-4f7f-9577-c985049b909a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:45.270929 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ec6f3-5f78-4f7f-9577-c985049b909a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:45.271233 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ec6f3-5f78-4f7f-9577-c985049b909a admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:45.271771 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ec6f3-5f78-4f7f-9577-c985049b909a admin admin] PATH_INFO: `/v3/projects/a5e0e0eab92e4036a0675d18e0c99391` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:45.273995 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ec6f3-5f78-4f7f-9577-c985049b909a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5e0e0eab92e4036a0675d18e0c99391)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:45.276117 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ec6f3-5f78-4f7f-9577-c985049b909a admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:45.335440 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-2d4ec6f3-5f78-4f7f-9577-c985049b909a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5e0e0eab92e4036a0675d18e0c99391', 'request_id': 'req-2d4ec6f3-5f78-4f7f-9577-c985049b909a'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:45.349062 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:39:45.354349 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-2d4ec6f3-5f78-4f7f-9577-c985049b909a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5e0e0eab92e4036a0675d18e0c99391', 'request_id': 'req-2d4ec6f3-5f78-4f7f-9577-c985049b909a'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:45.362070 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-2d4ec6f3-5f78-4f7f-9577-c985049b909a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5e0e0eab92e4036a0675d18e0c99391', 'request_id': 'req-2d4ec6f3-5f78-4f7f-9577-c985049b909a'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:45.373398 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-2d4ec6f3-5f78-4f7f-9577-c985049b909a admin admin] The token cache is being invalidate because project a5e0e0eab92e4036a0675d18e0c99391 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:39:45.373686 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-2d4ec6f3-5f78-4f7f-9577-c985049b909a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:45.377945 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-2d4ec6f3-5f78-4f7f-9577-c985049b909a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:45.379641 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1117/2226] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:39:45 2022] DELETE /identity/v3/projects/a5e0e0eab92e4036a0675d18e0c99391 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:39:45.385095 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4576e3-dc95-43b0-98ea-6ad803d46baa None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:45.385559 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4576e3-dc95-43b0-98ea-6ad803d46baa None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:45.385871 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4576e3-dc95-43b0-98ea-6ad803d46baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:45.403792 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-0c4576e3-dc95-43b0-98ea-6ad803d46baa None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:39:45.404983 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0c4576e3-dc95-43b0-98ea-6ad803d46baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:45.466806 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1110/2227] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:39:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:39:45.474627 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f6b90-31a9-4baa-abd8-82cd15699c0b None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:45.474916 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f6b90-31a9-4baa-abd8-82cd15699c0b None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:45.475199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f6b90-31a9-4baa-abd8-82cd15699c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:45.494961 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-b66f6b90-31a9-4baa-abd8-82cd15699c0b None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:39:45.496186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-b66f6b90-31a9-4baa-abd8-82cd15699c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:45.504904 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1118/2228] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:39:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:39:45.510013 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4576e3-dc95-43b0-98ea-6ad803d46baa None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:45.520909 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4576e3-dc95-43b0-98ea-6ad803d46baa None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:45.527060 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbfed7c-8dd3-422d-be93-0d96805c01ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:45.527723 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfed7c-8dd3-422d-be93-0d96805c01ee admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:45.528001 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfed7c-8dd3-422d-be93-0d96805c01ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:45.528295 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfed7c-8dd3-422d-be93-0d96805c01ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:45.529126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbfed7c-8dd3-422d-be93-0d96805c01ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:39:45.529488 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbfed7c-8dd3-422d-be93-0d96805c01ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:45.531347 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbfed7c-8dd3-422d-be93-0d96805c01ee admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:45.543805 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1111/2229] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:39:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:39:45.548188 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66f6b90-31a9-4baa-abd8-82cd15699c0b None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:45.558191 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66f6b90-31a9-4baa-abd8-82cd15699c0b None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:45.562938 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b14534-0be5-4acb-add4-9ef1c0f55eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:45.563590 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b14534-0be5-4acb-add4-9ef1c0f55eda admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:45.563877 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b14534-0be5-4acb-add4-9ef1c0f55eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:45.564155 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b14534-0be5-4acb-add4-9ef1c0f55eda admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:45.564856 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b14534-0be5-4acb-add4-9ef1c0f55eda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:45.566793 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b14534-0be5-4acb-add4-9ef1c0f55eda admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:45.591280 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1119/2230] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:39:45 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:39:45.595147 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbfed7c-8dd3-422d-be93-0d96805c01ee admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:45.605428 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbfed7c-8dd3-422d-be93-0d96805c01ee admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:45.646110 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:39:45.661580 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a5b50-d29f-4ff5-8ea9-6084f0111358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:45.662280 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a5b50-d29f-4ff5-8ea9-6084f0111358 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:45.662569 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a5b50-d29f-4ff5-8ea9-6084f0111358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:45.662849 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a5b50-d29f-4ff5-8ea9-6084f0111358 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:45.663550 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a5b50-d29f-4ff5-8ea9-6084f0111358 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:45.665972 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a5b50-d29f-4ff5-8ea9-6084f0111358 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:45.686280 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1112/2231] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:39:45 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:39:45.690198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b14534-0be5-4acb-add4-9ef1c0f55eda admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:45.700414 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b14534-0be5-4acb-add4-9ef1c0f55eda admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:45.704995 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662ec49-38d7-4e9b-bf44-a405d96298d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:45.705650 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662ec49-38d7-4e9b-bf44-a405d96298d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:45.705939 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662ec49-38d7-4e9b-bf44-a405d96298d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:45.706252 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662ec49-38d7-4e9b-bf44-a405d96298d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:45.706987 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4662ec49-38d7-4e9b-bf44-a405d96298d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:45.708730 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4662ec49-38d7-4e9b-bf44-a405d96298d7 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:45.719817 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1120/2232] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:39:45.723144 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a5b50-d29f-4ff5-8ea9-6084f0111358 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:45.733342 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a5b50-d29f-4ff5-8ea9-6084f0111358 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:45.738002 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e9ea1-7b6f-45d5-b111-ddce0ca3356d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:45.738684 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e9ea1-7b6f-45d5-b111-ddce0ca3356d admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:45.738967 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e9ea1-7b6f-45d5-b111-ddce0ca3356d admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:45.739239 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e9ea1-7b6f-45d5-b111-ddce0ca3356d admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:45.739959 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75e9ea1-7b6f-45d5-b111-ddce0ca3356d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:45.741535 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75e9ea1-7b6f-45d5-b111-ddce0ca3356d admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:45.753826 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1113/2233] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:39:45.757610 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662ec49-38d7-4e9b-bf44-a405d96298d7 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:45.767870 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662ec49-38d7-4e9b-bf44-a405d96298d7 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:45.772482 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29262a-467f-4317-bc22-863f79e6e861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:45.773351 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29262a-467f-4317-bc22-863f79e6e861 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:45.773638 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29262a-467f-4317-bc22-863f79e6e861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:45.773918 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29262a-467f-4317-bc22-863f79e6e861 admin admin] PATH_INFO: `/v3/projects/41993c05636f45ec8daaba106b5b1405/users/ea886098b54e401893f0c80cdcd9f4f7/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:45.788423 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a29262a-467f-4317-bc22-863f79e6e861 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41993c05636f45ec8daaba106b5b1405, user_id=ea886098b54e401893f0c80cdcd9f4f7, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:45.791623 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a29262a-467f-4317-bc22-863f79e6e861 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:45.804259 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1121/2234] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:39:45 2022] PUT /identity/v3/projects/41993c05636f45ec8daaba106b5b1405/users/ea886098b54e401893f0c80cdcd9f4f7/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:39:45.815624 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ab200-9dc5-4ed3-828d-d8bb2c33c9ba None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:45.815624 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ab200-9dc5-4ed3-828d-d8bb2c33c9ba None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:45.815919 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ab200-9dc5-4ed3-828d-d8bb2c33c9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:45.841812 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-741ab200-9dc5-4ed3-828d-d8bb2c33c9ba None None] MFA Rules not processed for user `ea886098b54e401893f0c80cdcd9f4f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:39:45.843047 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-741ab200-9dc5-4ed3-828d-d8bb2c33c9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:45.904217 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1114/2235] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:39:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:39:45.915279 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29262a-467f-4317-bc22-863f79e6e861 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:45.916926 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-8a29262a-467f-4317-bc22-863f79e6e861 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:45.972465 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29262a-467f-4317-bc22-863f79e6e861 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:45.976083 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13cb549-8b82-4e78-b2f4-5038bdd8f56b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:45.976725 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13cb549-8b82-4e78-b2f4-5038bdd8f56b service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:45.977611 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13cb549-8b82-4e78-b2f4-5038bdd8f56b service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:45.977885 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13cb549-8b82-4e78-b2f4-5038bdd8f56b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:45.986403 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13cb549-8b82-4e78-b2f4-5038bdd8f56b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:45.987821 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13cb549-8b82-4e78-b2f4-5038bdd8f56b service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:45.997733 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1122/2236] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:39:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:39:47.512146 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741ab200-9dc5-4ed3-828d-d8bb2c33c9ba None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:47.513653 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-741ab200-9dc5-4ed3-828d-d8bb2c33c9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:47.580418 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741ab200-9dc5-4ed3-828d-d8bb2c33c9ba None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:47.586011 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6f430-3010-4f1d-9dcb-672aec15593b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:47.587544 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6f430-3010-4f1d-9dcb-672aec15593b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:47.587544 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6f430-3010-4f1d-9dcb-672aec15593b admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:47.587544 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6f430-3010-4f1d-9dcb-672aec15593b admin admin] PATH_INFO: `/v3/users/a3cd96ddc54246629eabb2b6ea3b4dd2` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:47.588992 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6f430-3010-4f1d-9dcb-672aec15593b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3cd96ddc54246629eabb2b6ea3b4dd2)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:47.591057 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6f430-3010-4f1d-9dcb-672aec15593b admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:47.659438 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-aca6f430-3010-4f1d-9dcb-672aec15593b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3cd96ddc54246629eabb2b6ea3b4dd2'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:47.660349 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-aca6f430-3010-4f1d-9dcb-672aec15593b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3cd96ddc54246629eabb2b6ea3b4dd2'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:47.661029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-aca6f430-3010-4f1d-9dcb-672aec15593b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3cd96ddc54246629eabb2b6ea3b4dd2'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:47.667939 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:39:47.677584 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-aca6f430-3010-4f1d-9dcb-672aec15593b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3cd96ddc54246629eabb2b6ea3b4dd2'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:47.686681 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-aca6f430-3010-4f1d-9dcb-672aec15593b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3cd96ddc54246629eabb2b6ea3b4dd2'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:47.719488 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1115/2237] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:39:47 2022] DELETE /identity/v3/users/a3cd96ddc54246629eabb2b6ea3b4dd2 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:39:47.724357 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202f75e-017a-4f79-9ada-c73b612d2563 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:47.725309 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202f75e-017a-4f79-9ada-c73b612d2563 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:47.725732 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202f75e-017a-4f79-9ada-c73b612d2563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:47.751694 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-a202f75e-017a-4f79-9ada-c73b612d2563 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:39:47.753298 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-a202f75e-017a-4f79-9ada-c73b612d2563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:47.837609 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1123/2238] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:39:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:39:47.847508 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6f430-3010-4f1d-9dcb-672aec15593b admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:47.849010 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-aca6f430-3010-4f1d-9dcb-672aec15593b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:47.908306 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6f430-3010-4f1d-9dcb-672aec15593b admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:47.912530 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc5047-f680-4466-b556-9bc3711b67df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:47.913183 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc5047-f680-4466-b556-9bc3711b67df service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:47.913463 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc5047-f680-4466-b556-9bc3711b67df service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:47.913734 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc5047-f680-4466-b556-9bc3711b67df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:47.923623 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dc5047-f680-4466-b556-9bc3711b67df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:47.925053 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dc5047-f680-4466-b556-9bc3711b67df service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:47.934971 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1116/2239] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:39:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:39:48.200552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a202f75e-017a-4f79-9ada-c73b612d2563 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:48.201998 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-a202f75e-017a-4f79-9ada-c73b612d2563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:48.214951 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a202f75e-017a-4f79-9ada-c73b612d2563 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:48.219566 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d3de8c-3d37-470d-b24b-88757b0219d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:48.220445 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3de8c-3d37-470d-b24b-88757b0219d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:48.220727 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3de8c-3d37-470d-b24b-88757b0219d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:48.221009 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3de8c-3d37-470d-b24b-88757b0219d4 admin admin] PATH_INFO: `/v3/projects/9fb58d9251c54f00913c42a6bbf05050` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:48.222327 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d3de8c-3d37-470d-b24b-88757b0219d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fb58d9251c54f00913c42a6bbf05050)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:48.224514 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d3de8c-3d37-470d-b24b-88757b0219d4 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:48.236550 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:39:48.274242 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-15d3de8c-3d37-470d-b24b-88757b0219d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fb58d9251c54f00913c42a6bbf05050', 'request_id': 'req-15d3de8c-3d37-470d-b24b-88757b0219d4'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:48.281642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-15d3de8c-3d37-470d-b24b-88757b0219d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fb58d9251c54f00913c42a6bbf05050', 'request_id': 'req-15d3de8c-3d37-470d-b24b-88757b0219d4'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:48.288306 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-15d3de8c-3d37-470d-b24b-88757b0219d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fb58d9251c54f00913c42a6bbf05050', 'request_id': 'req-15d3de8c-3d37-470d-b24b-88757b0219d4'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:48.299252 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-15d3de8c-3d37-470d-b24b-88757b0219d4 admin admin] The token cache is being invalidate because project 9fb58d9251c54f00913c42a6bbf05050 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:39:48.299539 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-15d3de8c-3d37-470d-b24b-88757b0219d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:48.300177 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-15d3de8c-3d37-470d-b24b-88757b0219d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:48.301924 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1124/2240] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:39:48 2022] DELETE /identity/v3/projects/9fb58d9251c54f00913c42a6bbf05050 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:39:48.307319 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2296718-36db-4503-864b-a922690e04bd None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:48.307617 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2296718-36db-4503-864b-a922690e04bd None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:48.307900 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2296718-36db-4503-864b-a922690e04bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:48.325911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-e2296718-36db-4503-864b-a922690e04bd None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:39:48.327178 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e2296718-36db-4503-864b-a922690e04bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:48.390723 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1117/2241] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:39:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:39:48.398442 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34106d-65b9-4a0c-be85-5513f666d7ac None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:48.399266 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34106d-65b9-4a0c-be85-5513f666d7ac None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:48.399551 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34106d-65b9-4a0c-be85-5513f666d7ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:48.417987 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-fd34106d-65b9-4a0c-be85-5513f666d7ac None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:39:48.419229 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-fd34106d-65b9-4a0c-be85-5513f666d7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:48.427115 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1125/2242] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:39:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:39:48.431175 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2296718-36db-4503-864b-a922690e04bd None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:48.440958 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2296718-36db-4503-864b-a922690e04bd None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:48.446045 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5edccd1-1ed0-48ec-973a-34f8ef5484fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:48.446725 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edccd1-1ed0-48ec-973a-34f8ef5484fe admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:48.447010 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edccd1-1ed0-48ec-973a-34f8ef5484fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:48.447291 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edccd1-1ed0-48ec-973a-34f8ef5484fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:48.448023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5edccd1-1ed0-48ec-973a-34f8ef5484fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:39:48.448334 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5edccd1-1ed0-48ec-973a-34f8ef5484fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:48.449941 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5edccd1-1ed0-48ec-973a-34f8ef5484fe admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:48.461963 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1118/2243] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:39:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:39:48.466251 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd34106d-65b9-4a0c-be85-5513f666d7ac None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:48.477744 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd34106d-65b9-4a0c-be85-5513f666d7ac None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:48.482419 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b093e-9e41-4dc0-8d55-5a2ec6180bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:48.483065 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b093e-9e41-4dc0-8d55-5a2ec6180bf1 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:48.483347 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b093e-9e41-4dc0-8d55-5a2ec6180bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:48.483621 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b093e-9e41-4dc0-8d55-5a2ec6180bf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:48.484318 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b093e-9e41-4dc0-8d55-5a2ec6180bf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:48.486211 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b093e-9e41-4dc0-8d55-5a2ec6180bf1 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:48.511578 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1126/2244] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:39:48 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:39:48.516731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5edccd1-1ed0-48ec-973a-34f8ef5484fe admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:48.527528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5edccd1-1ed0-48ec-973a-34f8ef5484fe admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:48.582629 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5b1eaa-3165-4d4e-92de-84828a3db0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:48.583293 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b1eaa-3165-4d4e-92de-84828a3db0bc admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:48.583578 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b1eaa-3165-4d4e-92de-84828a3db0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:48.583854 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b1eaa-3165-4d4e-92de-84828a3db0bc admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:48.584539 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5b1eaa-3165-4d4e-92de-84828a3db0bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:48.587076 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5b1eaa-3165-4d4e-92de-84828a3db0bc admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:48.594597 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:39:48.611354 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1119/2245] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:39:48 2022] POST /identity/v3/users => generated 283 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:39:48.612512 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b093e-9e41-4dc0-8d55-5a2ec6180bf1 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:48.622754 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b093e-9e41-4dc0-8d55-5a2ec6180bf1 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:48.627614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea56ce-9b66-4e88-9927-b5863e53c175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:48.628261 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea56ce-9b66-4e88-9927-b5863e53c175 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:48.628552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea56ce-9b66-4e88-9927-b5863e53c175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:48.628829 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea56ce-9b66-4e88-9927-b5863e53c175 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:48.629587 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ea56ce-9b66-4e88-9927-b5863e53c175 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:48.631211 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ea56ce-9b66-4e88-9927-b5863e53c175 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:48.641961 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1127/2246] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:39:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:39:48.645698 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5b1eaa-3165-4d4e-92de-84828a3db0bc admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:48.655673 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5b1eaa-3165-4d4e-92de-84828a3db0bc admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:48.660236 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762673a-985c-4054-bfa3-8f8ac2897dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:48.660893 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762673a-985c-4054-bfa3-8f8ac2897dd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:48.661178 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762673a-985c-4054-bfa3-8f8ac2897dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:48.661455 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762673a-985c-4054-bfa3-8f8ac2897dd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:48.662241 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0762673a-985c-4054-bfa3-8f8ac2897dd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:48.663879 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0762673a-985c-4054-bfa3-8f8ac2897dd2 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:48.674253 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1120/2247] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:39:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:39:48.678208 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea56ce-9b66-4e88-9927-b5863e53c175 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:48.688502 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea56ce-9b66-4e88-9927-b5863e53c175 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:48.693084 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ab8f5-fc7f-4296-a6e1-c5a9acbab3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:48.693972 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ab8f5-fc7f-4296-a6e1-c5a9acbab3ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:48.694287 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ab8f5-fc7f-4296-a6e1-c5a9acbab3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:48.694567 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ab8f5-fc7f-4296-a6e1-c5a9acbab3ea admin admin] PATH_INFO: `/v3/projects/a89ba8be04784e9ea71660fa3e6e5f36/users/7b628a27482543ff953e7618041eba9c/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:48.708775 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8ab8f5-fc7f-4296-a6e1-c5a9acbab3ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=a89ba8be04784e9ea71660fa3e6e5f36, user_id=7b628a27482543ff953e7618041eba9c, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:48.711567 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8ab8f5-fc7f-4296-a6e1-c5a9acbab3ea admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:48.724419 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1128/2248] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:39:48 2022] PUT /identity/v3/projects/a89ba8be04784e9ea71660fa3e6e5f36/users/7b628a27482543ff953e7618041eba9c/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:39:48.735654 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22e875-ef29-4bf8-bde1-40159f91dfdd None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:48.735948 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22e875-ef29-4bf8-bde1-40159f91dfdd None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:48.736230 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22e875-ef29-4bf8-bde1-40159f91dfdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:48.762971 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-ce22e875-ef29-4bf8-bde1-40159f91dfdd None None] MFA Rules not processed for user `7b628a27482543ff953e7618041eba9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:39:48.764109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ce22e875-ef29-4bf8-bde1-40159f91dfdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:48.832714 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1121/2249] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:39:48 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:39:48.844541 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ab8f5-fc7f-4296-a6e1-c5a9acbab3ea admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:48.846268 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-8d8ab8f5-fc7f-4296-a6e1-c5a9acbab3ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:48.904534 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ab8f5-fc7f-4296-a6e1-c5a9acbab3ea admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:48.908226 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dcb39d-eb16-4009-b68b-6ab6fd5d79ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:48.908873 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcb39d-eb16-4009-b68b-6ab6fd5d79ca service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:48.909162 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcb39d-eb16-4009-b68b-6ab6fd5d79ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:48.909441 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcb39d-eb16-4009-b68b-6ab6fd5d79ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:48.919010 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dcb39d-eb16-4009-b68b-6ab6fd5d79ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:48.920460 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dcb39d-eb16-4009-b68b-6ab6fd5d79ca service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:48.929421 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1129/2250] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:39:48 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:39:51.872239 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e8801-66f9-4596-8eb7-cf633360e7dd None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:51.872745 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e8801-66f9-4596-8eb7-cf633360e7dd None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:51.872855 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e8801-66f9-4596-8eb7-cf633360e7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:51.891181 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-a70e8801-66f9-4596-8eb7-cf633360e7dd None None] MFA Rules not processed for user `e1635576dfe242f88ec1abb7be6359fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:39:51.892380 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a70e8801-66f9-4596-8eb7-cf633360e7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:51.949638 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1122/2251] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:39:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:39:56.144562 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dcb39d-eb16-4009-b68b-6ab6fd5d79ca service cinder] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:56.146651 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-35dcb39d-eb16-4009-b68b-6ab6fd5d79ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:56.199624 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:39:56.230738 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dcb39d-eb16-4009-b68b-6ab6fd5d79ca service cinder] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:56.240374 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52698a0a-1b7f-4970-b0a4-a51494d8c9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:56.241051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52698a0a-1b7f-4970-b0a4-a51494d8c9b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:56.241339 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52698a0a-1b7f-4970-b0a4-a51494d8c9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:56.241613 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52698a0a-1b7f-4970-b0a4-a51494d8c9b1 admin admin] PATH_INFO: `/v3/users/ea886098b54e401893f0c80cdcd9f4f7` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:56.243014 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52698a0a-1b7f-4970-b0a4-a51494d8c9b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea886098b54e401893f0c80cdcd9f4f7)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:56.245006 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52698a0a-1b7f-4970-b0a4-a51494d8c9b1 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:56.294575 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-52698a0a-1b7f-4970-b0a4-a51494d8c9b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea886098b54e401893f0c80cdcd9f4f7'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:56.295234 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-52698a0a-1b7f-4970-b0a4-a51494d8c9b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea886098b54e401893f0c80cdcd9f4f7'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:56.303181 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-52698a0a-1b7f-4970-b0a4-a51494d8c9b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea886098b54e401893f0c80cdcd9f4f7'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:56.303825 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-52698a0a-1b7f-4970-b0a4-a51494d8c9b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea886098b54e401893f0c80cdcd9f4f7'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:56.311966 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-52698a0a-1b7f-4970-b0a4-a51494d8c9b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea886098b54e401893f0c80cdcd9f4f7'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:56.331109 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1130/2252] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:39:56 2022] DELETE /identity/v3/users/ea886098b54e401893f0c80cdcd9f4f7 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:39:56.335709 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f37b2c-d44e-48b7-b11f-6126a7ed0baa None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:56.336203 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f37b2c-d44e-48b7-b11f-6126a7ed0baa None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:56.336283 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f37b2c-d44e-48b7-b11f-6126a7ed0baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:56.355525 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c6f37b2c-d44e-48b7-b11f-6126a7ed0baa None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:39:56.356796 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c6f37b2c-d44e-48b7-b11f-6126a7ed0baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:56.415881 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1123/2253] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:39:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:39:56.425648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52698a0a-1b7f-4970-b0a4-a51494d8c9b1 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:56.427064 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-52698a0a-1b7f-4970-b0a4-a51494d8c9b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:56.484753 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52698a0a-1b7f-4970-b0a4-a51494d8c9b1 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:56.488409 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e075fb77-92c9-416b-ac8a-186302e32410 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:56.489058 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075fb77-92c9-416b-ac8a-186302e32410 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:56.489343 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075fb77-92c9-416b-ac8a-186302e32410 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:56.489620 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075fb77-92c9-416b-ac8a-186302e32410 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:56.498668 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e075fb77-92c9-416b-ac8a-186302e32410 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:56.500361 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e075fb77-92c9-416b-ac8a-186302e32410 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:56.509820 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1131/2254] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:39:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:39:56.785355 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f37b2c-d44e-48b7-b11f-6126a7ed0baa None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:56.786852 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c6f37b2c-d44e-48b7-b11f-6126a7ed0baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:56.812824 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f37b2c-d44e-48b7-b11f-6126a7ed0baa None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:56.818242 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14f9df0-0782-4675-a6fa-86935d15b097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:56.819106 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f9df0-0782-4675-a6fa-86935d15b097 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:56.819386 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f9df0-0782-4675-a6fa-86935d15b097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:56.819655 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f9df0-0782-4675-a6fa-86935d15b097 admin admin] PATH_INFO: `/v3/projects/41993c05636f45ec8daaba106b5b1405` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:56.821375 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14f9df0-0782-4675-a6fa-86935d15b097 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41993c05636f45ec8daaba106b5b1405)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:56.824360 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14f9df0-0782-4675-a6fa-86935d15b097 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:56.838979 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:39:56.888204 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-d14f9df0-0782-4675-a6fa-86935d15b097 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41993c05636f45ec8daaba106b5b1405', 'request_id': 'req-d14f9df0-0782-4675-a6fa-86935d15b097'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:56.898205 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-d14f9df0-0782-4675-a6fa-86935d15b097 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41993c05636f45ec8daaba106b5b1405', 'request_id': 'req-d14f9df0-0782-4675-a6fa-86935d15b097'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:56.905629 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-d14f9df0-0782-4675-a6fa-86935d15b097 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41993c05636f45ec8daaba106b5b1405', 'request_id': 'req-d14f9df0-0782-4675-a6fa-86935d15b097'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:56.916479 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-d14f9df0-0782-4675-a6fa-86935d15b097 admin admin] The token cache is being invalidate because project 41993c05636f45ec8daaba106b5b1405 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:39:56.916775 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-d14f9df0-0782-4675-a6fa-86935d15b097 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:56.917420 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-d14f9df0-0782-4675-a6fa-86935d15b097 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:39:56.919187 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1124/2255] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:39:56 2022] DELETE /identity/v3/projects/41993c05636f45ec8daaba106b5b1405 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:39:56.923745 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6cf58-e35b-495c-b9a9-fa4b0383cfa9 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:56.924039 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6cf58-e35b-495c-b9a9-fa4b0383cfa9 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:56.924321 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6cf58-e35b-495c-b9a9-fa4b0383cfa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:56.942809 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-e6b6cf58-e35b-495c-b9a9-fa4b0383cfa9 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:39:56.944048 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-e6b6cf58-e35b-495c-b9a9-fa4b0383cfa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:57.005987 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1132/2256] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:39:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:39:57.015241 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065ff53-9cbf-447d-af4e-2075a3cf5153 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:57.015538 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065ff53-9cbf-447d-af4e-2075a3cf5153 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:57.015819 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065ff53-9cbf-447d-af4e-2075a3cf5153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:57.035357 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-6065ff53-9cbf-447d-af4e-2075a3cf5153 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:39:57.036728 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6065ff53-9cbf-447d-af4e-2075a3cf5153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:57.044926 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1125/2257] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:39:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:39:57.048922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b6cf58-e35b-495c-b9a9-fa4b0383cfa9 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:57.053689 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:39:57.061310 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b6cf58-e35b-495c-b9a9-fa4b0383cfa9 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:57.066930 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a548ae5d-843c-43b6-b1a6-a832cca66594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:57.067585 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548ae5d-843c-43b6-b1a6-a832cca66594 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:57.067866 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548ae5d-843c-43b6-b1a6-a832cca66594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:57.068138 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548ae5d-843c-43b6-b1a6-a832cca66594 admin admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:57.068867 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a548ae5d-843c-43b6-b1a6-a832cca66594 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:39:57.069174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a548ae5d-843c-43b6-b1a6-a832cca66594 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:57.070840 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a548ae5d-843c-43b6-b1a6-a832cca66594 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:57.086999 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1133/2258] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:39:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:39:57.088638 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065ff53-9cbf-447d-af4e-2075a3cf5153 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:57.100183 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065ff53-9cbf-447d-af4e-2075a3cf5153 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:57.106921 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce6f41-df90-420c-9578-366bd3f2a3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:57.108047 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce6f41-df90-420c-9578-366bd3f2a3e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:57.108420 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce6f41-df90-420c-9578-366bd3f2a3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:57.108721 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce6f41-df90-420c-9578-366bd3f2a3e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:57.109657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce6f41-df90-420c-9578-366bd3f2a3e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:57.111654 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce6f41-df90-420c-9578-366bd3f2a3e2 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:57.154144 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1126/2259] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:39:57 2022] POST /identity/v3/projects => generated 381 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:39:57.158175 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a548ae5d-843c-43b6-b1a6-a832cca66594 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:57.172214 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a548ae5d-843c-43b6-b1a6-a832cca66594 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:57.251627 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666ee31b-5d27-46c6-ab3b-4242d62c0433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:57.252381 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ee31b-5d27-46c6-ab3b-4242d62c0433 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:57.252741 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ee31b-5d27-46c6-ab3b-4242d62c0433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:57.253097 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ee31b-5d27-46c6-ab3b-4242d62c0433 admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:57.253883 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666ee31b-5d27-46c6-ab3b-4242d62c0433 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:57.255905 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666ee31b-5d27-46c6-ab3b-4242d62c0433 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:57.283039 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1134/2260] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:39:57 2022] POST /identity/v3/users => generated 291 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:39:57.287138 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce6f41-df90-420c-9578-366bd3f2a3e2 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:57.298843 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce6f41-df90-420c-9578-366bd3f2a3e2 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:57.303795 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7fcf8-ade8-40d5-a995-77739f5c373e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:57.304621 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7fcf8-ade8-40d5-a995-77739f5c373e admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:57.305454 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7fcf8-ade8-40d5-a995-77739f5c373e admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:57.305454 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7fcf8-ade8-40d5-a995-77739f5c373e admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:57.305972 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed7fcf8-ade8-40d5-a995-77739f5c373e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:57.307611 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed7fcf8-ade8-40d5-a995-77739f5c373e admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:57.319899 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1127/2261] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:39:57.323819 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666ee31b-5d27-46c6-ab3b-4242d62c0433 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:57.344474 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666ee31b-5d27-46c6-ab3b-4242d62c0433 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:57.349326 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74aafcd-afb8-48c6-a4fd-7a245c44d2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:57.349983 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74aafcd-afb8-48c6-a4fd-7a245c44d2ca admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:57.350299 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74aafcd-afb8-48c6-a4fd-7a245c44d2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:57.350574 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74aafcd-afb8-48c6-a4fd-7a245c44d2ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:57.351362 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74aafcd-afb8-48c6-a4fd-7a245c44d2ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:57.352977 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74aafcd-afb8-48c6-a4fd-7a245c44d2ca admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:57.364836 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1135/2262] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:39:57.369902 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7fcf8-ade8-40d5-a995-77739f5c373e admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:57.380506 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7fcf8-ade8-40d5-a995-77739f5c373e admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:57.386311 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea00ff49-8dd4-4dec-a5ab-0f809999d9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:57.387190 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00ff49-8dd4-4dec-a5ab-0f809999d9a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:57.387475 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00ff49-8dd4-4dec-a5ab-0f809999d9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:57.387750 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00ff49-8dd4-4dec-a5ab-0f809999d9a3 admin admin] PATH_INFO: `/v3/projects/526f86dc251440b9b50e325de4efb751/users/d6619b929f1844f48e0aa452ff5b8b51/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:57.404037 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea00ff49-8dd4-4dec-a5ab-0f809999d9a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=526f86dc251440b9b50e325de4efb751, user_id=d6619b929f1844f48e0aa452ff5b8b51, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:57.406872 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea00ff49-8dd4-4dec-a5ab-0f809999d9a3 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:57.425907 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1128/2263] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:39:57 2022] PUT /identity/v3/projects/526f86dc251440b9b50e325de4efb751/users/d6619b929f1844f48e0aa452ff5b8b51/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:39:57.427506 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859372bf-c65d-4d9a-a6da-b4a6419fb7c5 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:57.427844 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859372bf-c65d-4d9a-a6da-b4a6419fb7c5 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:57.428134 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859372bf-c65d-4d9a-a6da-b4a6419fb7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:57.447212 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-859372bf-c65d-4d9a-a6da-b4a6419fb7c5 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:39:57.448426 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-859372bf-c65d-4d9a-a6da-b4a6419fb7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:57.517081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1136/2264] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:39:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:39:57.527126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea00ff49-8dd4-4dec-a5ab-0f809999d9a3 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:57.529416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ea00ff49-8dd4-4dec-a5ab-0f809999d9a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:57.588257 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea00ff49-8dd4-4dec-a5ab-0f809999d9a3 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:57.591933 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee412e55-ff54-4b6d-b011-deea85bd9e1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:57.592582 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee412e55-ff54-4b6d-b011-deea85bd9e1a service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:57.592866 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee412e55-ff54-4b6d-b011-deea85bd9e1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:57.593142 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee412e55-ff54-4b6d-b011-deea85bd9e1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:57.602999 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee412e55-ff54-4b6d-b011-deea85bd9e1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:57.605346 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee412e55-ff54-4b6d-b011-deea85bd9e1a service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:57.616532 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1129/2265] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:39:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:39:58.281739 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4f79e-1d3e-4d90-bf16-6a32c9a4836c None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:58.282572 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4f79e-1d3e-4d90-bf16-6a32c9a4836c None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:58.282861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4f79e-1d3e-4d90-bf16-6a32c9a4836c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:58.311373 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-9cf4f79e-1d3e-4d90-bf16-6a32c9a4836c None None] MFA Rules not processed for user `d6619b929f1844f48e0aa452ff5b8b51`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:39:58.312594 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-9cf4f79e-1d3e-4d90-bf16-6a32c9a4836c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:58.404682 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:39:58.411468 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1137/2266] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:39:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:39:58.424795 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee412e55-ff54-4b6d-b011-deea85bd9e1a service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:39:58.426705 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ee412e55-ff54-4b6d-b011-deea85bd9e1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:58.493392 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:39:58.513749 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee412e55-ff54-4b6d-b011-deea85bd9e1a service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:39:58.517482 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e01d259-ed88-4f3b-a81c-15c9ffdff09a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:39:58.518285 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01d259-ed88-4f3b-a81c-15c9ffdff09a service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:58.518668 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01d259-ed88-4f3b-a81c-15c9ffdff09a service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:58.519066 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01d259-ed88-4f3b-a81c-15c9ffdff09a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:58.528957 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e01d259-ed88-4f3b-a81c-15c9ffdff09a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:39:58.530663 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e01d259-ed88-4f3b-a81c-15c9ffdff09a service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:39:58.540713 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1130/2267] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:39:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:39:59.871212 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4b681-03d0-4fae-858a-894f51d3c176 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:39:59.871704 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4b681-03d0-4fae-858a-894f51d3c176 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:39:59.871821 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4b681-03d0-4fae-858a-894f51d3c176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:39:59.890028 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-bba4b681-03d0-4fae-858a-894f51d3c176 None None] MFA Rules not processed for user `d6619b929f1844f48e0aa452ff5b8b51`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:39:59.891241 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-bba4b681-03d0-4fae-858a-894f51d3c176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:39:59.898211 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1138/2268] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:39:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:40:04.262805 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b48b8c-5e1a-4891-a5d2-8fcf593b935c None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:40:04.263645 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b48b8c-5e1a-4891-a5d2-8fcf593b935c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:40:04.264145 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b48b8c-5e1a-4891-a5d2-8fcf593b935c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:40:04.285734 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-41b48b8c-5e1a-4891-a5d2-8fcf593b935c None None] MFA Rules not processed for user `f26a832b69564f2a8387215afba6730e`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:40:04.287129 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-41b48b8c-5e1a-4891-a5d2-8fcf593b935c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:40:04.362360 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1131/2269] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:40:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:40:09.884912 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=4 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:40:14.354845 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=34 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:40:19.560656 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089e33a-1c28-4b8b-92d8-edf326707e2b None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:40:19.561295 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089e33a-1c28-4b8b-92d8-edf326707e2b None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:40:19.562022 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089e33a-1c28-4b8b-92d8-edf326707e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:40:19.593912 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-0089e33a-1c28-4b8b-92d8-edf326707e2b None None] MFA Rules not processed for user `67aa048d409c46e9a729aec9d1b2286e`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:40:19.595196 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-0089e33a-1c28-4b8b-92d8-edf326707e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:40:19.676027 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1139/2270] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Jun 15 19:40:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:40:19.723543 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b48b8c-5e1a-4891-a5d2-8fcf593b935c None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:40:19.725218 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-41b48b8c-5e1a-4891-a5d2-8fcf593b935c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:40:19.877209 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b48b8c-5e1a-4891-a5d2-8fcf593b935c None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:40:19.887233 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe13a49f-4b80-4e9a-82a4-4a9ff29883d7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176d5775a70f4524a9b7fe1ce9d426ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:40:19.887902 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13a49f-4b80-4e9a-82a4-4a9ff29883d7 service swift] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:40:19.888191 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13a49f-4b80-4e9a-82a4-4a9ff29883d7 service swift] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:40:19.888470 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13a49f-4b80-4e9a-82a4-4a9ff29883d7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:40:19.898038 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe13a49f-4b80-4e9a-82a4-4a9ff29883d7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:40:19.899824 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe13a49f-4b80-4e9a-82a4-4a9ff29883d7 service swift] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:40:19.911127 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1132/2271] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Jun 15 19:40:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 15 19:40:29.670647 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=38 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:40:29.907753 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=47 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:40:33.928829 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f77d3-4f21-40c2-9508-e24085f904ee None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:40:33.929377 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f77d3-4f21-40c2-9508-e24085f904ee None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:40:33.929377 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f77d3-4f21-40c2-9508-e24085f904ee None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:40:33.930824 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1140/2272] 10.222.0.15 () {54 vars in 921 bytes} [Wed Jun 15 19:40:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:40:33.937523 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1bf49-9dfb-4a4d-8a9e-6b4bc6f39057 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:40:33.937983 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1bf49-9dfb-4a4d-8a9e-6b4bc6f39057 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:40:33.938108 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1bf49-9dfb-4a4d-8a9e-6b4bc6f39057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:40:34.021031 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-8cf1bf49-9dfb-4a4d-8a9e-6b4bc6f39057 None None] MFA Rules not processed for user `3e74ac44e5814122a3226ac023f3688a`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:40:34.022424 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8cf1bf49-9dfb-4a4d-8a9e-6b4bc6f39057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:40:34.114929 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1133/2273] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Jun 15 19:40:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:40:34.127197 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f77d3-4f21-40c2-9508-e24085f904ee None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:40:34.128856 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-131f77d3-4f21-40c2-9508-e24085f904ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:40:34.188373 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f77d3-4f21-40c2-9508-e24085f904ee None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:40:34.191988 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a9ba2f-61e2-45a3-8c46-021e099ede31 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfca4d302aa43249609513f35d55d7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:40:34.192635 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a9ba2f-61e2-45a3-8c46-021e099ede31 service glance] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:40:34.192924 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a9ba2f-61e2-45a3-8c46-021e099ede31 service glance] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:40:34.193206 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a9ba2f-61e2-45a3-8c46-021e099ede31 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:40:34.203343 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a9ba2f-61e2-45a3-8c46-021e099ede31 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:40:34.204925 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a9ba2f-61e2-45a3-8c46-021e099ede31 service glance] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:40:34.215641 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1141/2274] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:40:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:40:34.280280 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447b921-a1b4-41e8-8396-803f2ffbcfa4 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:40:34.280510 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447b921-a1b4-41e8-8396-803f2ffbcfa4 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:40:34.280786 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447b921-a1b4-41e8-8396-803f2ffbcfa4 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:40:34.282227 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1134/2275] 10.222.0.15 () {54 vars in 921 bytes} [Wed Jun 15 19:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:40:34.287867 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c6102-0a7c-4fdc-911b-3a30f0430ec4 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:40:34.288164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c6102-0a7c-4fdc-911b-3a30f0430ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:40:34.288447 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c6102-0a7c-4fdc-911b-3a30f0430ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:40:34.318554 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-467c6102-0a7c-4fdc-911b-3a30f0430ec4 None None] MFA Rules not processed for user `5c63e0b6ceed4f7cb2e6f9269eb85795`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:40:34.319739 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-467c6102-0a7c-4fdc-911b-3a30f0430ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:40:34.335921 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1142/2276] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Jun 15 19:40:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:40:34.347677 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447b921-a1b4-41e8-8396-803f2ffbcfa4 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:40:34.349376 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b447b921-a1b4-41e8-8396-803f2ffbcfa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:40:34.363395 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447b921-a1b4-41e8-8396-803f2ffbcfa4 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:40:34.367521 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed171c1a-1c54-4bb3-ba9e-ef166bdda171 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:40:34.368150 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed171c1a-1c54-4bb3-ba9e-ef166bdda171 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:40:34.368431 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed171c1a-1c54-4bb3-ba9e-ef166bdda171 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:40:34.368701 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed171c1a-1c54-4bb3-ba9e-ef166bdda171 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:40:34.375857 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed171c1a-1c54-4bb3-ba9e-ef166bdda171 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:40:34.377272 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed171c1a-1c54-4bb3-ba9e-ef166bdda171 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:40:34.384916 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1135/2277] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:40:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:40:34.468237 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1416f97-64da-4a4b-a9dd-334a40aa520e None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:40:34.468532 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1416f97-64da-4a4b-a9dd-334a40aa520e None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:40:34.468809 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1416f97-64da-4a4b-a9dd-334a40aa520e None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:40:34.470210 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1143/2278] 10.222.0.15 () {54 vars in 921 bytes} [Wed Jun 15 19:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:40:34.475884 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb24c0-0605-47a4-a89d-2ca67941b4c0 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:40:34.476246 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb24c0-0605-47a4-a89d-2ca67941b4c0 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:40:34.476588 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb24c0-0605-47a4-a89d-2ca67941b4c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:40:34.495637 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-aefb24c0-0605-47a4-a89d-2ca67941b4c0 None None] MFA Rules not processed for user `3e74ac44e5814122a3226ac023f3688a`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:40:34.496843 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-aefb24c0-0605-47a4-a89d-2ca67941b4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:40:34.506896 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1136/2279] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Jun 15 19:40:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:40:34.517062 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1416f97-64da-4a4b-a9dd-334a40aa520e None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:40:34.527889 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1416f97-64da-4a4b-a9dd-334a40aa520e None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:40:34.531623 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c68eb-ee8d-4c92-9a4c-6b4ae167e4cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:40:34.532230 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c68eb-ee8d-4c92-9a4c-6b4ae167e4cc service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:40:34.532512 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c68eb-ee8d-4c92-9a4c-6b4ae167e4cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:40:34.532787 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c68eb-ee8d-4c92-9a4c-6b4ae167e4cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:40:34.544922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705c68eb-ee8d-4c92-9a4c-6b4ae167e4cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:40:34.546478 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705c68eb-ee8d-4c92-9a4c-6b4ae167e4cc service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:40:34.559204 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1144/2280] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:40:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:40:37.625898 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9614b19-fdb2-42f6-946b-e8ea4cd31deb None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:40:37.627632 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9614b19-fdb2-42f6-946b-e8ea4cd31deb None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:40:37.628505 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9614b19-fdb2-42f6-946b-e8ea4cd31deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:40:37.653918 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-a9614b19-fdb2-42f6-946b-e8ea4cd31deb None None] MFA Rules not processed for user `67aa048d409c46e9a729aec9d1b2286e`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:40:37.655221 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a9614b19-fdb2-42f6-946b-e8ea4cd31deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:40:37.666379 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1137/2281] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Jun 15 19:40:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:40:37.678973 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c68eb-ee8d-4c92-9a4c-6b4ae167e4cc service neutron] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:40:37.690877 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c68eb-ee8d-4c92-9a4c-6b4ae167e4cc service neutron] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:40:37.695928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc522e-ffc9-4630-9676-aab606e4bf32 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176d5775a70f4524a9b7fe1ce9d426ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:40:37.696640 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc522e-ffc9-4630-9676-aab606e4bf32 service swift] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:40:37.697044 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc522e-ffc9-4630-9676-aab606e4bf32 service swift] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:40:37.697510 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc522e-ffc9-4630-9676-aab606e4bf32 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:40:37.706877 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc522e-ffc9-4630-9676-aab606e4bf32 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:40:37.708473 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc522e-ffc9-4630-9676-aab606e4bf32 service swift] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:40:37.720643 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1145/2282] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Jun 15 19:40:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 19:40:39.609608 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9614b19-fdb2-42f6-946b-e8ea4cd31deb None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:40:39.611170 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a9614b19-fdb2-42f6-946b-e8ea4cd31deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:40:39.660153 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9614b19-fdb2-42f6-946b-e8ea4cd31deb None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:40:39.665185 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff04cac-3e06-4ace-b64d-c3717f59801e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:40:39.666573 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff04cac-3e06-4ace-b64d-c3717f59801e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:40:39.666861 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff04cac-3e06-4ace-b64d-c3717f59801e admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:40:39.667137 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff04cac-3e06-4ace-b64d-c3717f59801e admin admin] PATH_INFO: `/v3/users/7b628a27482543ff953e7618041eba9c` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:40:39.668528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff04cac-3e06-4ace-b64d-c3717f59801e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b628a27482543ff953e7618041eba9c)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:40:39.670672 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff04cac-3e06-4ace-b64d-c3717f59801e admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:40:39.737455 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:40:39.755070 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-aff04cac-3e06-4ace-b64d-c3717f59801e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b628a27482543ff953e7618041eba9c'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:40:39.756345 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-aff04cac-3e06-4ace-b64d-c3717f59801e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b628a27482543ff953e7618041eba9c'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:40:39.757026 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-aff04cac-3e06-4ace-b64d-c3717f59801e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b628a27482543ff953e7618041eba9c'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:40:39.772293 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-aff04cac-3e06-4ace-b64d-c3717f59801e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b628a27482543ff953e7618041eba9c'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:40:39.781079 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-aff04cac-3e06-4ace-b64d-c3717f59801e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b628a27482543ff953e7618041eba9c'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:40:39.805329 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1138/2283] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:40:39 2022] DELETE /identity/v3/users/7b628a27482543ff953e7618041eba9c => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:40:39.812928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05855854-79fc-4527-9c1e-8b0ea47d11cf None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:40:39.813895 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05855854-79fc-4527-9c1e-8b0ea47d11cf None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:40:39.814702 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05855854-79fc-4527-9c1e-8b0ea47d11cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:40:39.851756 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-05855854-79fc-4527-9c1e-8b0ea47d11cf None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:40:39.852895 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-05855854-79fc-4527-9c1e-8b0ea47d11cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:40:39.916675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1146/2284] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:40:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:40:39.926519 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff04cac-3e06-4ace-b64d-c3717f59801e admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:40:39.929224 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-aff04cac-3e06-4ace-b64d-c3717f59801e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:40:39.994875 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff04cac-3e06-4ace-b64d-c3717f59801e admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:40:39.999234 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3bd7d8-3180-4267-bfe9-072e990568da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:40:39.999911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bd7d8-3180-4267-bfe9-072e990568da service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:40:40.000374 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bd7d8-3180-4267-bfe9-072e990568da service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:40:40.000664 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bd7d8-3180-4267-bfe9-072e990568da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:40:40.012079 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3bd7d8-3180-4267-bfe9-072e990568da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:40:40.013835 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3bd7d8-3180-4267-bfe9-072e990568da service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:40:40.024632 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1139/2285] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:40:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:40:40.359291 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05855854-79fc-4527-9c1e-8b0ea47d11cf None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:40:40.361340 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-05855854-79fc-4527-9c1e-8b0ea47d11cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:40:40.398362 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05855854-79fc-4527-9c1e-8b0ea47d11cf None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:40:40.412934 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f18466c-6b6d-4c5c-bbe1-747be94e0297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:40:40.414712 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f18466c-6b6d-4c5c-bbe1-747be94e0297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:40:40.415514 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f18466c-6b6d-4c5c-bbe1-747be94e0297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:40:40.416072 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f18466c-6b6d-4c5c-bbe1-747be94e0297 admin admin] PATH_INFO: `/v3/projects/a89ba8be04784e9ea71660fa3e6e5f36` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:40:40.420051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f18466c-6b6d-4c5c-bbe1-747be94e0297 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a89ba8be04784e9ea71660fa3e6e5f36)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:40:40.424694 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f18466c-6b6d-4c5c-bbe1-747be94e0297 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:40:40.476132 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:40:40.487042 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-0f18466c-6b6d-4c5c-bbe1-747be94e0297 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a89ba8be04784e9ea71660fa3e6e5f36', 'request_id': 'req-0f18466c-6b6d-4c5c-bbe1-747be94e0297'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:40:40.497413 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-0f18466c-6b6d-4c5c-bbe1-747be94e0297 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a89ba8be04784e9ea71660fa3e6e5f36', 'request_id': 'req-0f18466c-6b6d-4c5c-bbe1-747be94e0297'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:40:40.506901 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-0f18466c-6b6d-4c5c-bbe1-747be94e0297 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a89ba8be04784e9ea71660fa3e6e5f36', 'request_id': 'req-0f18466c-6b6d-4c5c-bbe1-747be94e0297'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:40:40.519209 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-0f18466c-6b6d-4c5c-bbe1-747be94e0297 admin admin] The token cache is being invalidate because project a89ba8be04784e9ea71660fa3e6e5f36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:40:40.519497 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-0f18466c-6b6d-4c5c-bbe1-747be94e0297 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:40:40.520173 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-0f18466c-6b6d-4c5c-bbe1-747be94e0297 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:40:40.521780 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1147/2286] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:40:40 2022] DELETE /identity/v3/projects/a89ba8be04784e9ea71660fa3e6e5f36 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:40:50.019712 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=42,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:40:50.515750 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=10,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:40:56.868647 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3bd7d8-3180-4267-bfe9-072e990568da service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:40:56.870031 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ed3bd7d8-3180-4267-bfe9-072e990568da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:40:56.960944 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3bd7d8-3180-4267-bfe9-072e990568da service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:40:57.020345 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745d102-a89d-42f9-8c45-71433383ee0c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb0ec2ef6c4748e181f1963e7dfe30b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:40:57.021008 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745d102-a89d-42f9-8c45-71433383ee0c service placement] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:40:57.021294 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745d102-a89d-42f9-8c45-71433383ee0c service placement] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:40:57.021574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745d102-a89d-42f9-8c45-71433383ee0c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:40:57.023627 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e745d102-a89d-42f9-8c45-71433383ee0c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:40:57.035112 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e745d102-a89d-42f9-8c45-71433383ee0c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:40:57.037236 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e745d102-a89d-42f9-8c45-71433383ee0c service placement] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:40:57.047280 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1140/2287] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:40:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:41:07.042958 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=67 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:41:10.262327 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93117236-c274-4184-9bd9-83a28294c9a9 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:41:10.262801 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93117236-c274-4184-9bd9-83a28294c9a9 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:41:10.262916 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93117236-c274-4184-9bd9-83a28294c9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:41:10.310831 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-93117236-c274-4184-9bd9-83a28294c9a9 None None] MFA Rules not processed for user `67aa048d409c46e9a729aec9d1b2286e`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:41:10.312041 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-93117236-c274-4184-9bd9-83a28294c9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:41:10.375702 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1148/2288] 10.222.0.15 () {58 vars in 967 bytes} [Wed Jun 15 19:41:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:41:10.389226 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745d102-a89d-42f9-8c45-71433383ee0c service placement] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:41:10.390711 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e745d102-a89d-42f9-8c45-71433383ee0c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:41:10.484845 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745d102-a89d-42f9-8c45-71433383ee0c service placement] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:41:10.490027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd658c72-10ab-426a-b357-e73d20a896f9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176d5775a70f4524a9b7fe1ce9d426ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:41:10.491421 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd658c72-10ab-426a-b357-e73d20a896f9 service swift] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:41:10.491701 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd658c72-10ab-426a-b357-e73d20a896f9 service swift] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:41:10.491972 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd658c72-10ab-426a-b357-e73d20a896f9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:41:10.501129 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd658c72-10ab-426a-b357-e73d20a896f9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:41:10.502751 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd658c72-10ab-426a-b357-e73d20a896f9 service swift] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:41:10.512079 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1141/2289] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Jun 15 19:41:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 19:41:20.372785 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=40 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:41:20.509358 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=41 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:41:26.544216 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82f1e3-4f09-4b85-ba47-32a7645d90ce None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:41:26.544854 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82f1e3-4f09-4b85-ba47-32a7645d90ce None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:41:26.544854 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82f1e3-4f09-4b85-ba47-32a7645d90ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:41:26.564001 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-3d82f1e3-4f09-4b85-ba47-32a7645d90ce None None] MFA Rules not processed for user `d6619b929f1844f48e0aa452ff5b8b51`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:41:26.565362 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-3d82f1e3-4f09-4b85-ba47-32a7645d90ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:41:26.626175 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1149/2290] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:41:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:41:36.617779 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=25 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:41:47.253801 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c1233-3295-4f0a-adef-b7d7ca9c0bff None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:41:47.254469 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c1233-3295-4f0a-adef-b7d7ca9c0bff None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:41:47.254614 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c1233-3295-4f0a-adef-b7d7ca9c0bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:41:47.281667 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-448c1233-3295-4f0a-adef-b7d7ca9c0bff None None] MFA Rules not processed for user `e1635576dfe242f88ec1abb7be6359fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:41:47.283658 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-448c1233-3295-4f0a-adef-b7d7ca9c0bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:41:47.363374 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1142/2291] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:41:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:41:57.351940 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=25 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:42:05.678138 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9290b-afcd-4b4f-8a7e-87aef9c536ae None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:42:05.679275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9290b-afcd-4b4f-8a7e-87aef9c536ae None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:42:05.679374 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9290b-afcd-4b4f-8a7e-87aef9c536ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:42:05.698082 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-fad9290b-afcd-4b4f-8a7e-87aef9c536ae None None] MFA Rules not processed for user `f26a832b69564f2a8387215afba6730e`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:42:05.699436 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-fad9290b-afcd-4b4f-8a7e-87aef9c536ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:42:05.758934 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1150/2292] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:42:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:42:06.750083 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42165cb1-29ab-4fc8-97db-887ad7345442 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:42:06.750615 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42165cb1-29ab-4fc8-97db-887ad7345442 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:42:06.750615 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42165cb1-29ab-4fc8-97db-887ad7345442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:42:06.770350 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-42165cb1-29ab-4fc8-97db-887ad7345442 None None] MFA Rules not processed for user `b2103a7ed6824444a60fa5573e7ad39e`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:42:06.771511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-42165cb1-29ab-4fc8-97db-887ad7345442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:42:06.857775 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1143/2293] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:42:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:42:15.751481 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=25 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:42:16.850699 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=25 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:42:30.641775 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad9290b-afcd-4b4f-8a7e-87aef9c536ae None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:42:30.643248 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-fad9290b-afcd-4b4f-8a7e-87aef9c536ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:42:30.712152 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad9290b-afcd-4b4f-8a7e-87aef9c536ae None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:42:30.716730 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6de9879-ea72-4570-97e4-76030704537f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:42:30.717612 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de9879-ea72-4570-97e4-76030704537f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:42:30.717897 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de9879-ea72-4570-97e4-76030704537f admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:42:30.718211 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de9879-ea72-4570-97e4-76030704537f admin admin] PATH_INFO: `/v3/users/d6619b929f1844f48e0aa452ff5b8b51` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:42:30.719418 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6de9879-ea72-4570-97e4-76030704537f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6619b929f1844f48e0aa452ff5b8b51)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:42:30.721352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6de9879-ea72-4570-97e4-76030704537f admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:42:30.773421 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-b6de9879-ea72-4570-97e4-76030704537f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6619b929f1844f48e0aa452ff5b8b51'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:42:30.774074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-b6de9879-ea72-4570-97e4-76030704537f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6619b929f1844f48e0aa452ff5b8b51'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:42:30.781629 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-b6de9879-ea72-4570-97e4-76030704537f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6619b929f1844f48e0aa452ff5b8b51'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:42:30.782294 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-b6de9879-ea72-4570-97e4-76030704537f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6619b929f1844f48e0aa452ff5b8b51'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:42:30.789669 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-b6de9879-ea72-4570-97e4-76030704537f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6619b929f1844f48e0aa452ff5b8b51'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:42:30.806674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1151/2294] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:42:30 2022] DELETE /identity/v3/users/d6619b929f1844f48e0aa452ff5b8b51 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:42:30.982308 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42165cb1-29ab-4fc8-97db-887ad7345442 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:42:30.996134 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42165cb1-29ab-4fc8-97db-887ad7345442 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:42:31.054827 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e13078-c7df-4d96-a6e7-2a47a04f2d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:42:31.055803 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e13078-c7df-4d96-a6e7-2a47a04f2d0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:42:31.056094 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e13078-c7df-4d96-a6e7-2a47a04f2d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:42:31.056372 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e13078-c7df-4d96-a6e7-2a47a04f2d0f admin admin] PATH_INFO: `/v3/projects/526f86dc251440b9b50e325de4efb751` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:42:31.057566 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e13078-c7df-4d96-a6e7-2a47a04f2d0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=526f86dc251440b9b50e325de4efb751)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:42:31.059554 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e13078-c7df-4d96-a6e7-2a47a04f2d0f admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:42:31.106290 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f7e13078-c7df-4d96-a6e7-2a47a04f2d0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '526f86dc251440b9b50e325de4efb751', 'request_id': 'req-f7e13078-c7df-4d96-a6e7-2a47a04f2d0f'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:42:31.113374 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f7e13078-c7df-4d96-a6e7-2a47a04f2d0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '526f86dc251440b9b50e325de4efb751', 'request_id': 'req-f7e13078-c7df-4d96-a6e7-2a47a04f2d0f'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:42:31.120497 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f7e13078-c7df-4d96-a6e7-2a47a04f2d0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '526f86dc251440b9b50e325de4efb751', 'request_id': 'req-f7e13078-c7df-4d96-a6e7-2a47a04f2d0f'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:42:31.132384 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f7e13078-c7df-4d96-a6e7-2a47a04f2d0f admin admin] The token cache is being invalidate because project 526f86dc251440b9b50e325de4efb751 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:42:31.132669 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f7e13078-c7df-4d96-a6e7-2a47a04f2d0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:42:31.133327 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f7e13078-c7df-4d96-a6e7-2a47a04f2d0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:42:31.135225 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1144/2295] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:42:30 2022] DELETE /identity/v3/projects/526f86dc251440b9b50e325de4efb751 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:42:40.802953 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:42:41.128489 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:42:42.152476 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554dc921-d3f0-424b-99cb-bc05b842862d None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:42:42.153009 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554dc921-d3f0-424b-99cb-bc05b842862d None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:42:42.153278 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554dc921-d3f0-424b-99cb-bc05b842862d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:42:42.172358 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-554dc921-d3f0-424b-99cb-bc05b842862d None None] MFA Rules not processed for user `c815423179f44b219e4f8572b10355c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:42:42.173576 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-554dc921-d3f0-424b-99cb-bc05b842862d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:42:42.232001 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1152/2296] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:42:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:42:46.579324 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e13078-c7df-4d96-a6e7-2a47a04f2d0f admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:42:46.581090 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f7e13078-c7df-4d96-a6e7-2a47a04f2d0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:42:46.684085 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e13078-c7df-4d96-a6e7-2a47a04f2d0f admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:42:46.687941 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1fd2f-1a9b-43de-87bf-91abaae7a21b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:42:46.688599 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1fd2f-1a9b-43de-87bf-91abaae7a21b service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:42:46.688879 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1fd2f-1a9b-43de-87bf-91abaae7a21b service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:42:46.689166 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1fd2f-1a9b-43de-87bf-91abaae7a21b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:42:46.691150 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d2d1fd2f-1a9b-43de-87bf-91abaae7a21b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:42:46.788435 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d1fd2f-1a9b-43de-87bf-91abaae7a21b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:42:46.790042 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d1fd2f-1a9b-43de-87bf-91abaae7a21b service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:42:46.801739 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1145/2297] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:42:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:42:49.603526 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554dc921-d3f0-424b-99cb-bc05b842862d None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:42:49.605090 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-554dc921-d3f0-424b-99cb-bc05b842862d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:42:49.666559 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554dc921-d3f0-424b-99cb-bc05b842862d None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:42:49.672561 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700930e7-25dc-41a6-b226-e0a584146090 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:42:49.673254 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700930e7-25dc-41a6-b226-e0a584146090 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:42:49.673589 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700930e7-25dc-41a6-b226-e0a584146090 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:42:49.673937 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700930e7-25dc-41a6-b226-e0a584146090 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:42:49.676061 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-700930e7-25dc-41a6-b226-e0a584146090 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:42:49.691253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700930e7-25dc-41a6-b226-e0a584146090 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:42:49.692733 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700930e7-25dc-41a6-b226-e0a584146090 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:42:49.704489 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1153/2298] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:42:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:42:56.796453 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=63 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:42:59.699330 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=51 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:43:03.360886 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1fd2f-1a9b-43de-87bf-91abaae7a21b service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:03.362420 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d2d1fd2f-1a9b-43de-87bf-91abaae7a21b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:03.380064 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1fd2f-1a9b-43de-87bf-91abaae7a21b service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:03.388740 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33882ce1-479a-4f74-82d6-5f60d19b42f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:03.389428 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33882ce1-479a-4f74-82d6-5f60d19b42f2 service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:03.389711 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33882ce1-479a-4f74-82d6-5f60d19b42f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:03.389985 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33882ce1-479a-4f74-82d6-5f60d19b42f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:03.393488 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-33882ce1-479a-4f74-82d6-5f60d19b42f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:03.463990 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33882ce1-479a-4f74-82d6-5f60d19b42f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:03.465829 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33882ce1-479a-4f74-82d6-5f60d19b42f2 service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:03.476209 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1146/2299] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:43:03 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:43:04.749220 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700930e7-25dc-41a6-b226-e0a584146090 service neutron] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:04.761169 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700930e7-25dc-41a6-b226-e0a584146090 service neutron] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:04.767301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba89edf-a0bd-4f01-939e-892a8227f6eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:04.767944 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba89edf-a0bd-4f01-939e-892a8227f6eb service nova] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:04.768226 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba89edf-a0bd-4f01-939e-892a8227f6eb service nova] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:04.768500 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba89edf-a0bd-4f01-939e-892a8227f6eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:04.771181 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-6ba89edf-a0bd-4f01-939e-892a8227f6eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:04.830544 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba89edf-a0bd-4f01-939e-892a8227f6eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:04.832670 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba89edf-a0bd-4f01-939e-892a8227f6eb service nova] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:04.842468 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1154/2300] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:43:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:43:10.985467 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33882ce1-479a-4f74-82d6-5f60d19b42f2 service nova] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:10.987056 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-33882ce1-479a-4f74-82d6-5f60d19b42f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:11.003873 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33882ce1-479a-4f74-82d6-5f60d19b42f2 service nova] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:11.009284 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6798f5c-202c-4d77-9095-15dbda2256db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:11.010518 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6798f5c-202c-4d77-9095-15dbda2256db service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:11.011023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6798f5c-202c-4d77-9095-15dbda2256db service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:11.011509 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6798f5c-202c-4d77-9095-15dbda2256db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:11.014421 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b6798f5c-202c-4d77-9095-15dbda2256db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:11.116419 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6798f5c-202c-4d77-9095-15dbda2256db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:11.119204 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6798f5c-202c-4d77-9095-15dbda2256db service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:11.134821 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba89edf-a0bd-4f01-939e-892a8227f6eb service nova] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:11.138490 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1147/2301] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:43:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:43:11.155334 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba89edf-a0bd-4f01-939e-892a8227f6eb service nova] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:11.163581 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b079c-6119-4afb-835a-0e117b45326a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:11.164354 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b079c-6119-4afb-835a-0e117b45326a service nova] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:11.164726 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b079c-6119-4afb-835a-0e117b45326a service nova] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:11.165091 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b079c-6119-4afb-835a-0e117b45326a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:11.167589 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-6d9b079c-6119-4afb-835a-0e117b45326a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:11.250976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b079c-6119-4afb-835a-0e117b45326a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:11.252657 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b079c-6119-4afb-835a-0e117b45326a service nova] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:11.265164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1155/2302] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:43:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:43:11.444818 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6798f5c-202c-4d77-9095-15dbda2256db service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:11.447027 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b6798f5c-202c-4d77-9095-15dbda2256db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:11.480569 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6798f5c-202c-4d77-9095-15dbda2256db service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:11.514179 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802de879-0894-4057-b51c-d50b0c69324c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:11.515136 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802de879-0894-4057-b51c-d50b0c69324c service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:11.515634 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802de879-0894-4057-b51c-d50b0c69324c service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:11.516112 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802de879-0894-4057-b51c-d50b0c69324c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:11.520609 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-802de879-0894-4057-b51c-d50b0c69324c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:11.537658 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802de879-0894-4057-b51c-d50b0c69324c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:11.539663 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802de879-0894-4057-b51c-d50b0c69324c service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:11.554942 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1148/2303] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:43:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:43:13.944398 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6420cd-889e-4566-9756-ab14987c899e None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:13.944993 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6420cd-889e-4566-9756-ab14987c899e None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:13.945507 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6420cd-889e-4566-9756-ab14987c899e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:13.973068 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-5c6420cd-889e-4566-9756-ab14987c899e None None] MFA Rules not processed for user `e1635576dfe242f88ec1abb7be6359fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:43:13.974978 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-5c6420cd-889e-4566-9756-ab14987c899e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:13.990237 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1156/2304] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:43:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:43:14.717481 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802de879-0894-4057-b51c-d50b0c69324c service nova] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:14.731285 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802de879-0894-4057-b51c-d50b0c69324c service nova] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:14.736466 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1245bb1-c6a7-4b78-8b9a-06dce653927f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:14.737135 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1245bb1-c6a7-4b78-8b9a-06dce653927f service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:14.737415 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1245bb1-c6a7-4b78-8b9a-06dce653927f service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:14.738589 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1245bb1-c6a7-4b78-8b9a-06dce653927f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:14.740608 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d1245bb1-c6a7-4b78-8b9a-06dce653927f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:14.754317 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1245bb1-c6a7-4b78-8b9a-06dce653927f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:14.756489 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1245bb1-c6a7-4b78-8b9a-06dce653927f service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:14.767411 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1149/2305] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:43:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:43:15.250970 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6420cd-889e-4566-9756-ab14987c899e None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:15.252288 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-5c6420cd-889e-4566-9756-ab14987c899e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:15.350946 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6420cd-889e-4566-9756-ab14987c899e None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:15.357033 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d2866-221a-43d3-81ee-aa2f82c92497 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb0ec2ef6c4748e181f1963e7dfe30b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:15.357904 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d2866-221a-43d3-81ee-aa2f82c92497 service placement] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:15.358460 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d2866-221a-43d3-81ee-aa2f82c92497 service placement] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:15.358642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d2866-221a-43d3-81ee-aa2f82c92497 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:15.361130 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-fc1d2866-221a-43d3-81ee-aa2f82c92497 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:15.378293 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1d2866-221a-43d3-81ee-aa2f82c92497 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:15.380596 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1d2866-221a-43d3-81ee-aa2f82c92497 service placement] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:15.392379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1157/2306] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:43:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:43:15.549393 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e912ee-cbda-4528-a3e8-8c115b11385e None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:15.551391 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e912ee-cbda-4528-a3e8-8c115b11385e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:15.552259 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e912ee-cbda-4528-a3e8-8c115b11385e None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:15.557756 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1150/2307] 10.222.0.15 () {54 vars in 921 bytes} [Wed Jun 15 19:43:15 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:43:15.591286 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a0317-8f33-4c70-89b4-4fdc26085f7b None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:15.591539 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a0317-8f33-4c70-89b4-4fdc26085f7b None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:15.591914 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a0317-8f33-4c70-89b4-4fdc26085f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:15.617021 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e912ee-cbda-4528-a3e8-8c115b11385e None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:15.619729 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b5e912ee-cbda-4528-a3e8-8c115b11385e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:15.645553 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e912ee-cbda-4528-a3e8-8c115b11385e None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:15.653426 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:43:15.653684 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ef2e2-9f1b-4f4f-9bc1-5454892ff329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:15.655079 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ef2e2-9f1b-4f4f-9bc1-5454892ff329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:15.655400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ef2e2-9f1b-4f4f-9bc1-5454892ff329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:15.655979 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ef2e2-9f1b-4f4f-9bc1-5454892ff329 admin admin] PATH_INFO: `/v3/users/f26a832b69564f2a8387215afba6730e` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:15.657910 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ef2e2-9f1b-4f4f-9bc1-5454892ff329 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f26a832b69564f2a8387215afba6730e)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:15.658061 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-108a0317-8f33-4c70-89b4-4fdc26085f7b None None] MFA Rules not processed for user `cb0ec2ef6c4748e181f1963e7dfe30b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:43:15.659550 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-108a0317-8f33-4c70-89b4-4fdc26085f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:15.661129 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ef2e2-9f1b-4f4f-9bc1-5454892ff329 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:15.679021 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1158/2308] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Jun 15 19:43:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:43:15.691494 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108a0317-8f33-4c70-89b4-4fdc26085f7b None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:15.692907 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-108a0317-8f33-4c70-89b4-4fdc26085f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:15.707827 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108a0317-8f33-4c70-89b4-4fdc26085f7b None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:15.716419 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57edf6af-44f1-4778-ae20-c0d415aaca65 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb0ec2ef6c4748e181f1963e7dfe30b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:15.717309 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57edf6af-44f1-4778-ae20-c0d415aaca65 service placement] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:15.717750 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57edf6af-44f1-4778-ae20-c0d415aaca65 service placement] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:15.718304 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57edf6af-44f1-4778-ae20-c0d415aaca65 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:15.730190 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57edf6af-44f1-4778-ae20-c0d415aaca65 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:15.732494 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57edf6af-44f1-4778-ae20-c0d415aaca65 service placement] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:15.748495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1159/2309] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:43:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:43:15.759724 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6c2ef2e2-9f1b-4f4f-9bc1-5454892ff329 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f26a832b69564f2a8387215afba6730e'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:15.766901 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6c2ef2e2-9f1b-4f4f-9bc1-5454892ff329 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f26a832b69564f2a8387215afba6730e'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:15.769147 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6c2ef2e2-9f1b-4f4f-9bc1-5454892ff329 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f26a832b69564f2a8387215afba6730e'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:15.796567 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:43:15.800221 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6c2ef2e2-9f1b-4f4f-9bc1-5454892ff329 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f26a832b69564f2a8387215afba6730e'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:15.816810 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6c2ef2e2-9f1b-4f4f-9bc1-5454892ff329 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f26a832b69564f2a8387215afba6730e'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:15.832176 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1151/2310] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:43:15 2022] DELETE /identity/v3/users/f26a832b69564f2a8387215afba6730e => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:43:16.024996 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57edf6af-44f1-4778-ae20-c0d415aaca65 service placement] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:16.039506 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57edf6af-44f1-4778-ae20-c0d415aaca65 service placement] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:16.113192 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379c6345-04ad-4de7-9c5a-a3c970333be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:16.114073 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c6345-04ad-4de7-9c5a-a3c970333be3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:16.114390 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c6345-04ad-4de7-9c5a-a3c970333be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:16.114669 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c6345-04ad-4de7-9c5a-a3c970333be3 admin admin] PATH_INFO: `/v3/projects/0d5270cd64fd4929af7b063cf2e2b653` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:16.115916 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379c6345-04ad-4de7-9c5a-a3c970333be3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d5270cd64fd4929af7b063cf2e2b653)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:16.117835 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379c6345-04ad-4de7-9c5a-a3c970333be3 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:16.167878 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-379c6345-04ad-4de7-9c5a-a3c970333be3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d5270cd64fd4929af7b063cf2e2b653', 'request_id': 'req-379c6345-04ad-4de7-9c5a-a3c970333be3'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:16.176025 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-379c6345-04ad-4de7-9c5a-a3c970333be3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d5270cd64fd4929af7b063cf2e2b653', 'request_id': 'req-379c6345-04ad-4de7-9c5a-a3c970333be3'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:16.182804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-379c6345-04ad-4de7-9c5a-a3c970333be3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d5270cd64fd4929af7b063cf2e2b653', 'request_id': 'req-379c6345-04ad-4de7-9c5a-a3c970333be3'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:16.194689 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-379c6345-04ad-4de7-9c5a-a3c970333be3 admin admin] The token cache is being invalidate because project 0d5270cd64fd4929af7b063cf2e2b653 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:43:16.194918 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-379c6345-04ad-4de7-9c5a-a3c970333be3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:16.195539 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-379c6345-04ad-4de7-9c5a-a3c970333be3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:16.197129 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1160/2311] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:43:16 2022] DELETE /identity/v3/projects/0d5270cd64fd4929af7b063cf2e2b653 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:43:16.203789 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a066cd-144b-4f9a-9207-851ac3da2003 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:16.204285 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a066cd-144b-4f9a-9207-851ac3da2003 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:16.204772 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a066cd-144b-4f9a-9207-851ac3da2003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:16.244706 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-d3a066cd-144b-4f9a-9207-851ac3da2003 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:43:16.246229 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d3a066cd-144b-4f9a-9207-851ac3da2003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:16.306136 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1152/2312] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:43:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:43:16.315926 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6910fa1f-7c4d-43cc-a9fc-926fb4360991 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:16.316406 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6910fa1f-7c4d-43cc-a9fc-926fb4360991 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:16.316802 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6910fa1f-7c4d-43cc-a9fc-926fb4360991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:16.352494 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-6910fa1f-7c4d-43cc-a9fc-926fb4360991 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:43:16.353801 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-6910fa1f-7c4d-43cc-a9fc-926fb4360991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:16.366550 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1161/2313] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:43:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:43:16.368455 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a066cd-144b-4f9a-9207-851ac3da2003 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:16.379107 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a066cd-144b-4f9a-9207-851ac3da2003 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:16.385343 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8bbad-b095-449b-9145-89e77f8c909b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:16.386116 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8bbad-b095-449b-9145-89e77f8c909b admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:16.386495 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8bbad-b095-449b-9145-89e77f8c909b admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:16.386878 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8bbad-b095-449b-9145-89e77f8c909b admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:16.387695 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8bbad-b095-449b-9145-89e77f8c909b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:43:16.388074 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8bbad-b095-449b-9145-89e77f8c909b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:16.389773 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8bbad-b095-449b-9145-89e77f8c909b admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:16.403400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1153/2314] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:43:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:43:16.407654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6910fa1f-7c4d-43cc-a9fc-926fb4360991 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:16.422893 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6910fa1f-7c4d-43cc-a9fc-926fb4360991 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:16.423794 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19269b7-6879-4562-82dd-da0fe1ee3585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:16.424491 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19269b7-6879-4562-82dd-da0fe1ee3585 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:16.424776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19269b7-6879-4562-82dd-da0fe1ee3585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:16.425054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19269b7-6879-4562-82dd-da0fe1ee3585 admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:16.425753 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19269b7-6879-4562-82dd-da0fe1ee3585 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:16.427696 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19269b7-6879-4562-82dd-da0fe1ee3585 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:16.457536 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1162/2315] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:43:16 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:43:16.461454 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8bbad-b095-449b-9145-89e77f8c909b admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:16.472053 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8bbad-b095-449b-9145-89e77f8c909b admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:16.548964 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c627696-58ca-41e7-af4c-10f42a2fdde3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:16.549972 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c627696-58ca-41e7-af4c-10f42a2fdde3 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:16.551005 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c627696-58ca-41e7-af4c-10f42a2fdde3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:16.551339 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c627696-58ca-41e7-af4c-10f42a2fdde3 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:16.552066 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c627696-58ca-41e7-af4c-10f42a2fdde3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:16.554377 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c627696-58ca-41e7-af4c-10f42a2fdde3 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:16.579446 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1154/2316] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:43:16 2022] POST /identity/v3/users => generated 288 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:43:16.584746 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19269b7-6879-4562-82dd-da0fe1ee3585 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:16.599092 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19269b7-6879-4562-82dd-da0fe1ee3585 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:16.604813 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c879ad-5de6-4032-b9c7-44b2aff24621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:16.605643 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c879ad-5de6-4032-b9c7-44b2aff24621 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:16.605933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c879ad-5de6-4032-b9c7-44b2aff24621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:16.606244 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c879ad-5de6-4032-b9c7-44b2aff24621 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:16.607485 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c879ad-5de6-4032-b9c7-44b2aff24621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:16.608589 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c879ad-5de6-4032-b9c7-44b2aff24621 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:16.623067 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1163/2317] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:43:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:43:16.627334 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c627696-58ca-41e7-af4c-10f42a2fdde3 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:16.637908 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c627696-58ca-41e7-af4c-10f42a2fdde3 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:16.643218 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aef6f8-be66-4ac6-b772-4bbb8a04f452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:16.643895 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aef6f8-be66-4ac6-b772-4bbb8a04f452 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:16.644184 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aef6f8-be66-4ac6-b772-4bbb8a04f452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:16.644461 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aef6f8-be66-4ac6-b772-4bbb8a04f452 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:16.645264 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aef6f8-be66-4ac6-b772-4bbb8a04f452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:16.647072 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aef6f8-be66-4ac6-b772-4bbb8a04f452 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:16.657860 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1155/2318] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:43:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:43:16.663210 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c879ad-5de6-4032-b9c7-44b2aff24621 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:16.673322 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c879ad-5de6-4032-b9c7-44b2aff24621 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:16.678970 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e012edee-bdcd-4fa7-a488-44d5d00da98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:16.679849 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012edee-bdcd-4fa7-a488-44d5d00da98b admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:16.680129 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012edee-bdcd-4fa7-a488-44d5d00da98b admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:16.680406 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012edee-bdcd-4fa7-a488-44d5d00da98b admin admin] PATH_INFO: `/v3/projects/7776863ddeb74fb28b1b8ea207cfa1da/users/ee9987a4b9e84333aa49dad2628165eb/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:16.696325 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e012edee-bdcd-4fa7-a488-44d5d00da98b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7776863ddeb74fb28b1b8ea207cfa1da, user_id=ee9987a4b9e84333aa49dad2628165eb, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:16.699010 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e012edee-bdcd-4fa7-a488-44d5d00da98b admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:16.712732 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1164/2319] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:43:16 2022] PUT /identity/v3/projects/7776863ddeb74fb28b1b8ea207cfa1da/users/ee9987a4b9e84333aa49dad2628165eb/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:43:16.723478 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4897a-23ea-4706-be8c-d59f04c6d4ae None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:16.723765 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4897a-23ea-4706-be8c-d59f04c6d4ae None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:16.724035 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4897a-23ea-4706-be8c-d59f04c6d4ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:16.753245 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-8be4897a-23ea-4706-be8c-d59f04c6d4ae None None] MFA Rules not processed for user `ee9987a4b9e84333aa49dad2628165eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:43:16.754834 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8be4897a-23ea-4706-be8c-d59f04c6d4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:16.775205 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:43:16.827321 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1156/2320] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:43:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:43:16.839096 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e012edee-bdcd-4fa7-a488-44d5d00da98b admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:16.840912 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-e012edee-bdcd-4fa7-a488-44d5d00da98b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:16.857409 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:43:16.928398 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e012edee-bdcd-4fa7-a488-44d5d00da98b admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:16.934435 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e9a2e-0426-4b30-b267-8416bc3b47e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:16.935104 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e9a2e-0426-4b30-b267-8416bc3b47e1 service nova] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:16.935396 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e9a2e-0426-4b30-b267-8416bc3b47e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:16.935679 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e9a2e-0426-4b30-b267-8416bc3b47e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:16.946771 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e9a2e-0426-4b30-b267-8416bc3b47e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:16.948379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e9a2e-0426-4b30-b267-8416bc3b47e1 service nova] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:16.959265 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1165/2321] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:43:16 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:43:17.188445 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4897a-23ea-4706-be8c-d59f04c6d4ae None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:17.256886 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4897a-23ea-4706-be8c-d59f04c6d4ae None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:17.261498 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1486ed-a442-4c4c-8208-ae3c7a22fafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:17.262551 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1486ed-a442-4c4c-8208-ae3c7a22fafd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:17.262839 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1486ed-a442-4c4c-8208-ae3c7a22fafd admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:17.263116 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1486ed-a442-4c4c-8208-ae3c7a22fafd admin admin] PATH_INFO: `/v3/users/ee9987a4b9e84333aa49dad2628165eb` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:17.264365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1486ed-a442-4c4c-8208-ae3c7a22fafd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee9987a4b9e84333aa49dad2628165eb)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:17.266732 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1486ed-a442-4c4c-8208-ae3c7a22fafd admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:17.344365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5f1486ed-a442-4c4c-8208-ae3c7a22fafd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee9987a4b9e84333aa49dad2628165eb'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:17.345931 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5f1486ed-a442-4c4c-8208-ae3c7a22fafd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee9987a4b9e84333aa49dad2628165eb'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:17.346316 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5f1486ed-a442-4c4c-8208-ae3c7a22fafd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee9987a4b9e84333aa49dad2628165eb'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:17.359977 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5f1486ed-a442-4c4c-8208-ae3c7a22fafd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee9987a4b9e84333aa49dad2628165eb'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:17.368425 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5f1486ed-a442-4c4c-8208-ae3c7a22fafd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee9987a4b9e84333aa49dad2628165eb'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:17.384485 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1157/2322] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:43:17 2022] DELETE /identity/v3/users/ee9987a4b9e84333aa49dad2628165eb => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:43:17.389124 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75052775-9a21-40bd-af28-e2a0d80f3f4b None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:17.389661 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75052775-9a21-40bd-af28-e2a0d80f3f4b None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:17.389861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75052775-9a21-40bd-af28-e2a0d80f3f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:17.395716 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1486ed-a442-4c4c-8208-ae3c7a22fafd admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:17.397481 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-5f1486ed-a442-4c4c-8208-ae3c7a22fafd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:17.409147 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-75052775-9a21-40bd-af28-e2a0d80f3f4b None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:43:17.411174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-75052775-9a21-40bd-af28-e2a0d80f3f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:17.451282 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:43:17.474392 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1486ed-a442-4c4c-8208-ae3c7a22fafd admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:17.479337 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d4685-ee00-47ba-9b36-06ad20f891fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:17.480456 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d4685-ee00-47ba-9b36-06ad20f891fa service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:17.480819 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d4685-ee00-47ba-9b36-06ad20f891fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:17.481165 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d4685-ee00-47ba-9b36-06ad20f891fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:17.483698 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d30d4685-ee00-47ba-9b36-06ad20f891fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:17.494367 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1166/2323] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:43:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:43:17.496054 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30d4685-ee00-47ba-9b36-06ad20f891fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:17.497539 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30d4685-ee00-47ba-9b36-06ad20f891fa service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:17.505271 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75052775-9a21-40bd-af28-e2a0d80f3f4b None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:17.508652 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1158/2324] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:43:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:43:17.517394 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75052775-9a21-40bd-af28-e2a0d80f3f4b None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:17.522340 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd71d61-64a3-4e41-a6e7-1b368658879c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:17.523000 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd71d61-64a3-4e41-a6e7-1b368658879c service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:17.523370 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd71d61-64a3-4e41-a6e7-1b368658879c service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:17.523679 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd71d61-64a3-4e41-a6e7-1b368658879c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:17.535692 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd71d61-64a3-4e41-a6e7-1b368658879c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:17.537869 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd71d61-64a3-4e41-a6e7-1b368658879c service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:17.551748 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1167/2325] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:43:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:43:17.992204 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d4685-ee00-47ba-9b36-06ad20f891fa service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:17.993869 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d30d4685-ee00-47ba-9b36-06ad20f891fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:18.012190 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d4685-ee00-47ba-9b36-06ad20f891fa service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:18.018274 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83d486-8a46-4e5d-bbf4-8551debecc71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:18.019265 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83d486-8a46-4e5d-bbf4-8551debecc71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:18.019607 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83d486-8a46-4e5d-bbf4-8551debecc71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:18.019962 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83d486-8a46-4e5d-bbf4-8551debecc71 admin admin] PATH_INFO: `/v3/projects/7776863ddeb74fb28b1b8ea207cfa1da` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:18.021988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a83d486-8a46-4e5d-bbf4-8551debecc71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7776863ddeb74fb28b1b8ea207cfa1da)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:18.024838 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a83d486-8a46-4e5d-bbf4-8551debecc71 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:18.083672 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4a83d486-8a46-4e5d-bbf4-8551debecc71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7776863ddeb74fb28b1b8ea207cfa1da', 'request_id': 'req-4a83d486-8a46-4e5d-bbf4-8551debecc71'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:18.095463 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4a83d486-8a46-4e5d-bbf4-8551debecc71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7776863ddeb74fb28b1b8ea207cfa1da', 'request_id': 'req-4a83d486-8a46-4e5d-bbf4-8551debecc71'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:18.105119 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4a83d486-8a46-4e5d-bbf4-8551debecc71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7776863ddeb74fb28b1b8ea207cfa1da', 'request_id': 'req-4a83d486-8a46-4e5d-bbf4-8551debecc71'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:18.128396 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4a83d486-8a46-4e5d-bbf4-8551debecc71 admin admin] The token cache is being invalidate because project 7776863ddeb74fb28b1b8ea207cfa1da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:43:18.128700 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4a83d486-8a46-4e5d-bbf4-8551debecc71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:18.129444 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4a83d486-8a46-4e5d-bbf4-8551debecc71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:18.134683 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1159/2326] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:43:17 2022] DELETE /identity/v3/projects/7776863ddeb74fb28b1b8ea207cfa1da => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:43:18.136836 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d143581-0a84-418a-85d9-da62f57f0eca None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:18.137265 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d143581-0a84-418a-85d9-da62f57f0eca None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:18.137663 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d143581-0a84-418a-85d9-da62f57f0eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:18.168454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-1d143581-0a84-418a-85d9-da62f57f0eca None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:43:18.169687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-1d143581-0a84-418a-85d9-da62f57f0eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:18.279579 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1168/2327] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:43:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:43:18.281536 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa522c-5670-423f-a954-6d05b940f402 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:18.281904 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa522c-5670-423f-a954-6d05b940f402 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:18.282148 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa522c-5670-423f-a954-6d05b940f402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:18.324355 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c5fa522c-5670-423f-a954-6d05b940f402 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:43:18.328193 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c5fa522c-5670-423f-a954-6d05b940f402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:18.369201 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1160/2328] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:43:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:43:18.370904 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d143581-0a84-418a-85d9-da62f57f0eca None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:18.381953 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d143581-0a84-418a-85d9-da62f57f0eca None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:18.405666 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d46eae-4291-46c9-8136-5c8585d95971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:18.406403 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d46eae-4291-46c9-8136-5c8585d95971 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:18.406689 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d46eae-4291-46c9-8136-5c8585d95971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:18.406969 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d46eae-4291-46c9-8136-5c8585d95971 admin admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:18.407734 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d46eae-4291-46c9-8136-5c8585d95971 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:43:18.408102 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d46eae-4291-46c9-8136-5c8585d95971 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:18.409814 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d46eae-4291-46c9-8136-5c8585d95971 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:18.433083 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1169/2329] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:43:18.434386 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fa522c-5670-423f-a954-6d05b940f402 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:18.445431 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fa522c-5670-423f-a954-6d05b940f402 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:18.450786 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a487f-faa4-46a1-b66e-4a98b4d39968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:18.451473 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a487f-faa4-46a1-b66e-4a98b4d39968 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:18.451800 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a487f-faa4-46a1-b66e-4a98b4d39968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:18.452088 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a487f-faa4-46a1-b66e-4a98b4d39968 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:18.452798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a487f-faa4-46a1-b66e-4a98b4d39968 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:18.455006 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a487f-faa4-46a1-b66e-4a98b4d39968 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:18.483223 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1161/2330] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:43:18 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:43:18.487265 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d46eae-4291-46c9-8136-5c8585d95971 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:18.500196 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d46eae-4291-46c9-8136-5c8585d95971 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:18.515172 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:43:18.568687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af131df-5215-4022-a614-814736df523b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:18.569365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af131df-5215-4022-a614-814736df523b admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:18.569651 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af131df-5215-4022-a614-814736df523b admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:18.569928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af131df-5215-4022-a614-814736df523b admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:18.570673 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af131df-5215-4022-a614-814736df523b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:18.572623 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af131df-5215-4022-a614-814736df523b admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:18.594798 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1170/2331] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:43:18 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:43:18.598913 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a487f-faa4-46a1-b66e-4a98b4d39968 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:18.609910 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a487f-faa4-46a1-b66e-4a98b4d39968 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:18.615882 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc52ed-2c77-40f0-99ed-b803dc9348e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:18.616665 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc52ed-2c77-40f0-99ed-b803dc9348e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:18.617221 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc52ed-2c77-40f0-99ed-b803dc9348e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:18.617433 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc52ed-2c77-40f0-99ed-b803dc9348e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:18.618306 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdc52ed-2c77-40f0-99ed-b803dc9348e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:18.637893 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdc52ed-2c77-40f0-99ed-b803dc9348e0 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:18.658609 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1162/2332] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:43:18.663717 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af131df-5215-4022-a614-814736df523b admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:18.675319 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af131df-5215-4022-a614-814736df523b admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:18.680536 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb71495-39f8-4cb3-8df1-10c941413b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:18.681211 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb71495-39f8-4cb3-8df1-10c941413b4d admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:18.681517 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb71495-39f8-4cb3-8df1-10c941413b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:18.682066 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb71495-39f8-4cb3-8df1-10c941413b4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:18.682849 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb71495-39f8-4cb3-8df1-10c941413b4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:18.685000 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb71495-39f8-4cb3-8df1-10c941413b4d admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:18.698125 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1171/2333] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:43:18.700631 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc52ed-2c77-40f0-99ed-b803dc9348e0 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:18.711790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc52ed-2c77-40f0-99ed-b803dc9348e0 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:18.719311 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d097ce-6423-46d2-9953-684b0a8e1f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:18.720383 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d097ce-6423-46d2-9953-684b0a8e1f2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:18.720680 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d097ce-6423-46d2-9953-684b0a8e1f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:18.720962 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d097ce-6423-46d2-9953-684b0a8e1f2d admin admin] PATH_INFO: `/v3/projects/65b89c7fe92745328d9c55540d20612b/users/4318211933d34371a4679e58da8c2eec/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:18.737728 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d097ce-6423-46d2-9953-684b0a8e1f2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=65b89c7fe92745328d9c55540d20612b, user_id=4318211933d34371a4679e58da8c2eec, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:18.740557 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d097ce-6423-46d2-9953-684b0a8e1f2d admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:18.768529 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1163/2334] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:43:18 2022] PUT /identity/v3/projects/65b89c7fe92745328d9c55540d20612b/users/4318211933d34371a4679e58da8c2eec/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:43:18.772260 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e62f7-a914-4169-a292-7f1412b04be8 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:18.772679 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e62f7-a914-4169-a292-7f1412b04be8 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:18.773036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e62f7-a914-4169-a292-7f1412b04be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:18.806104 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-560e62f7-a914-4169-a292-7f1412b04be8 None None] MFA Rules not processed for user `4318211933d34371a4679e58da8c2eec`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:43:18.807350 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-560e62f7-a914-4169-a292-7f1412b04be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:18.874963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1172/2335] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:43:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:43:18.879948 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d097ce-6423-46d2-9953-684b0a8e1f2d admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:18.940613 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d097ce-6423-46d2-9953-684b0a8e1f2d admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:18.946519 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be749f-63ca-4a29-9907-5d0140d5f791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:18.947210 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be749f-63ca-4a29-9907-5d0140d5f791 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:18.947498 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be749f-63ca-4a29-9907-5d0140d5f791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:18.947778 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be749f-63ca-4a29-9907-5d0140d5f791 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:18.948482 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be749f-63ca-4a29-9907-5d0140d5f791 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:18.950468 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be749f-63ca-4a29-9907-5d0140d5f791 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:18.953657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:43:18.977127 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1164/2336] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:43:18 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:43:18.981189 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e62f7-a914-4169-a292-7f1412b04be8 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:18.991849 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e62f7-a914-4169-a292-7f1412b04be8 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:19.053660 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09de861-b10f-4aad-b242-5b56bfd1293c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:19.054277 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09de861-b10f-4aad-b242-5b56bfd1293c admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:19.054564 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09de861-b10f-4aad-b242-5b56bfd1293c admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:19.054837 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09de861-b10f-4aad-b242-5b56bfd1293c admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:19.055526 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09de861-b10f-4aad-b242-5b56bfd1293c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:19.057428 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09de861-b10f-4aad-b242-5b56bfd1293c admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:19.080874 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1173/2337] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:43:18 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:43:19.085516 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be749f-63ca-4a29-9907-5d0140d5f791 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:19.096231 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be749f-63ca-4a29-9907-5d0140d5f791 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:19.101441 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047aa52-4c3d-4eeb-8b10-62fa8e306484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:19.102133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047aa52-4c3d-4eeb-8b10-62fa8e306484 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:19.102415 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047aa52-4c3d-4eeb-8b10-62fa8e306484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:19.102691 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047aa52-4c3d-4eeb-8b10-62fa8e306484 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:19.103418 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047aa52-4c3d-4eeb-8b10-62fa8e306484 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:19.105019 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047aa52-4c3d-4eeb-8b10-62fa8e306484 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:19.116223 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1165/2338] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:43:19.122761 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09de861-b10f-4aad-b242-5b56bfd1293c admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:19.138849 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09de861-b10f-4aad-b242-5b56bfd1293c admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:19.143558 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d454a08-9782-4777-8238-4a986d1dbcc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:19.144440 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d454a08-9782-4777-8238-4a986d1dbcc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:19.144725 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d454a08-9782-4777-8238-4a986d1dbcc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:19.145008 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d454a08-9782-4777-8238-4a986d1dbcc4 admin admin] PATH_INFO: `/v3/projects/477a6028011f44c68183f514c509eabc/users/3422598c40864e5cb13f5457bf71ebdb/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:19.153812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:43:19.161460 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d454a08-9782-4777-8238-4a986d1dbcc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=477a6028011f44c68183f514c509eabc, user_id=3422598c40864e5cb13f5457bf71ebdb, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:19.164162 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d454a08-9782-4777-8238-4a986d1dbcc4 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:19.177349 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1174/2339] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:43:19 2022] PUT /identity/v3/projects/477a6028011f44c68183f514c509eabc/users/3422598c40864e5cb13f5457bf71ebdb/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:43:19.181366 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047aa52-4c3d-4eeb-8b10-62fa8e306484 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:19.192111 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047aa52-4c3d-4eeb-8b10-62fa8e306484 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:19.250528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1041e760-5122-4b99-bd8a-992ce0005d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:19.251174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041e760-5122-4b99-bd8a-992ce0005d55 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:19.251476 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041e760-5122-4b99-bd8a-992ce0005d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:19.251749 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041e760-5122-4b99-bd8a-992ce0005d55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:19.252477 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1041e760-5122-4b99-bd8a-992ce0005d55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:19.254696 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1041e760-5122-4b99-bd8a-992ce0005d55 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:19.264749 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1166/2340] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:43:19.268840 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d454a08-9782-4777-8238-4a986d1dbcc4 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:19.279033 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d454a08-9782-4777-8238-4a986d1dbcc4 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:19.283567 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ee2597-37f4-4d29-acf7-e3377b2b547c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:19.284486 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee2597-37f4-4d29-acf7-e3377b2b547c admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:19.284765 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee2597-37f4-4d29-acf7-e3377b2b547c admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:19.285035 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee2597-37f4-4d29-acf7-e3377b2b547c admin admin] PATH_INFO: `/v3/projects/477a6028011f44c68183f514c509eabc/users/3422598c40864e5cb13f5457bf71ebdb/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:19.287119 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ee2597-37f4-4d29-acf7-e3377b2b547c admin admin] RBAC: Authorizing `identity:create_grant(project_id=477a6028011f44c68183f514c509eabc, user_id=3422598c40864e5cb13f5457bf71ebdb, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:19.289730 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ee2597-37f4-4d29-acf7-e3377b2b547c admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:19.299700 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1175/2341] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:43:19 2022] PUT /identity/v3/projects/477a6028011f44c68183f514c509eabc/users/3422598c40864e5cb13f5457bf71ebdb/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:43:19.310222 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d8e06-a5f7-4776-b597-980e44cb6213 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:19.310446 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d8e06-a5f7-4776-b597-980e44cb6213 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:19.310719 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d8e06-a5f7-4776-b597-980e44cb6213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:19.337287 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-a91d8e06-a5f7-4776-b597-980e44cb6213 None None] MFA Rules not processed for user `3422598c40864e5cb13f5457bf71ebdb`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:43:19.338520 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a91d8e06-a5f7-4776-b597-980e44cb6213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:19.408211 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1167/2342] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:43:19 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:43:19.419418 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ee2597-37f4-4d29-acf7-e3377b2b547c admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:19.421109 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-67ee2597-37f4-4d29-acf7-e3377b2b547c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:19.481172 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ee2597-37f4-4d29-acf7-e3377b2b547c admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:19.484958 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a60fbd-b837-4b22-be29-5082b93541fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:19.485605 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a60fbd-b837-4b22-be29-5082b93541fc service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:19.485887 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a60fbd-b837-4b22-be29-5082b93541fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:19.486189 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a60fbd-b837-4b22-be29-5082b93541fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:19.546697 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a60fbd-b837-4b22-be29-5082b93541fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:19.548140 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a60fbd-b837-4b22-be29-5082b93541fc service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:19.557714 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1176/2343] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:43:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:43:24.185552 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d8e06-a5f7-4776-b597-980e44cb6213 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:24.198029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d8e06-a5f7-4776-b597-980e44cb6213 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:24.208945 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3663b36-afef-416d-b751-590dc14d0d3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:24.209620 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3663b36-afef-416d-b751-590dc14d0d3e service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:24.209907 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3663b36-afef-416d-b751-590dc14d0d3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:24.210288 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3663b36-afef-416d-b751-590dc14d0d3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:24.226901 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3663b36-afef-416d-b751-590dc14d0d3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:24.229803 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3663b36-afef-416d-b751-590dc14d0d3e service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:24.249635 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1168/2344] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:43:24 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:43:26.714810 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a60fbd-b837-4b22-be29-5082b93541fc service cinder] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:26.716332 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-74a60fbd-b837-4b22-be29-5082b93541fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:26.794000 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a60fbd-b837-4b22-be29-5082b93541fc service cinder] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:26.799512 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb9451-dda8-46e3-872c-5ddc4e76940a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:26.800189 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb9451-dda8-46e3-872c-5ddc4e76940a service nova] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:26.800472 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb9451-dda8-46e3-872c-5ddc4e76940a service nova] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:26.800749 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb9451-dda8-46e3-872c-5ddc4e76940a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:26.802542 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-57bb9451-dda8-46e3-872c-5ddc4e76940a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:26.814170 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bb9451-dda8-46e3-872c-5ddc4e76940a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:26.815772 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bb9451-dda8-46e3-872c-5ddc4e76940a service nova] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:26.826067 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1177/2345] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:43:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:43:30.532645 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3663b36-afef-416d-b751-590dc14d0d3e service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:30.545281 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3663b36-afef-416d-b751-590dc14d0d3e service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:30.551875 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af3743-50f6-43a8-ac48-90e35ab30a63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:30.552372 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af3743-50f6-43a8-ac48-90e35ab30a63 service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:30.552655 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af3743-50f6-43a8-ac48-90e35ab30a63 service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:30.552933 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af3743-50f6-43a8-ac48-90e35ab30a63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:30.554783 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e1af3743-50f6-43a8-ac48-90e35ab30a63 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:30.569187 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af3743-50f6-43a8-ac48-90e35ab30a63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:30.570841 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af3743-50f6-43a8-ac48-90e35ab30a63 service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:30.581555 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1169/2346] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:43:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:43:34.573472 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf78a6-48dd-416d-b43e-1e822ee391ef None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:34.574016 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf78a6-48dd-416d-b43e-1e822ee391ef None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:34.574016 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf78a6-48dd-416d-b43e-1e822ee391ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:34.594415 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-3fbf78a6-48dd-416d-b43e-1e822ee391ef None None] MFA Rules not processed for user `67aa048d409c46e9a729aec9d1b2286e`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:43:34.595686 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-3fbf78a6-48dd-416d-b43e-1e822ee391ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:34.608045 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:43:34.663275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1178/2347] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Jun 15 19:43:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:43:34.713284 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af3743-50f6-43a8-ac48-90e35ab30a63 service nova] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:34.714905 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e1af3743-50f6-43a8-ac48-90e35ab30a63 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:34.749024 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:43:34.789696 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af3743-50f6-43a8-ac48-90e35ab30a63 service nova] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:34.794827 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f445f-001a-4b64-8ae8-be2d8892e169 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176d5775a70f4524a9b7fe1ce9d426ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:34.795479 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f445f-001a-4b64-8ae8-be2d8892e169 service swift] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:34.795766 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f445f-001a-4b64-8ae8-be2d8892e169 service swift] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:34.796048 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f445f-001a-4b64-8ae8-be2d8892e169 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:34.805306 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9f445f-001a-4b64-8ae8-be2d8892e169 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:34.806988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9f445f-001a-4b64-8ae8-be2d8892e169 service swift] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:34.819362 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1170/2348] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Jun 15 19:43:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 19:43:42.064273 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf78a6-48dd-416d-b43e-1e822ee391ef None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:42.078416 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf78a6-48dd-416d-b43e-1e822ee391ef None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:42.084746 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e177b-137d-478c-bb13-f057eaac940b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:42.085584 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e177b-137d-478c-bb13-f057eaac940b service nova] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:42.085811 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e177b-137d-478c-bb13-f057eaac940b service nova] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:42.086118 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e177b-137d-478c-bb13-f057eaac940b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:42.088027 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-f05e177b-137d-478c-bb13-f057eaac940b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:42.179390 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e177b-137d-478c-bb13-f057eaac940b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:42.181286 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e177b-137d-478c-bb13-f057eaac940b service nova] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:42.207020 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1179/2349] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:43:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:43:44.812168 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=19 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:43:49.535755 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f445f-001a-4b64-8ae8-be2d8892e169 service swift] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:49.614695 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f445f-001a-4b64-8ae8-be2d8892e169 service swift] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:49.620371 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dddaa1-bf98-4e0e-bc3e-d6b375803a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:49.621450 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dddaa1-bf98-4e0e-bc3e-d6b375803a65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:49.621877 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dddaa1-bf98-4e0e-bc3e-d6b375803a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:49.622621 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dddaa1-bf98-4e0e-bc3e-d6b375803a65 admin admin] PATH_INFO: `/v3/users/4318211933d34371a4679e58da8c2eec` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:49.624067 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dddaa1-bf98-4e0e-bc3e-d6b375803a65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4318211933d34371a4679e58da8c2eec)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:49.626884 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dddaa1-bf98-4e0e-bc3e-d6b375803a65 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:49.711015 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-81dddaa1-bf98-4e0e-bc3e-d6b375803a65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4318211933d34371a4679e58da8c2eec'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:49.711675 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-81dddaa1-bf98-4e0e-bc3e-d6b375803a65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4318211933d34371a4679e58da8c2eec'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:49.712365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-81dddaa1-bf98-4e0e-bc3e-d6b375803a65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4318211933d34371a4679e58da8c2eec'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:49.735190 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-81dddaa1-bf98-4e0e-bc3e-d6b375803a65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4318211933d34371a4679e58da8c2eec'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:49.751516 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-81dddaa1-bf98-4e0e-bc3e-d6b375803a65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4318211933d34371a4679e58da8c2eec'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:49.772734 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1171/2350] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:43:49 2022] DELETE /identity/v3/users/4318211933d34371a4679e58da8c2eec => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:43:49.777298 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ef0c7-60e9-41ec-bbbd-cd57c1f2e0c8 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:49.777867 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ef0c7-60e9-41ec-bbbd-cd57c1f2e0c8 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:49.777867 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ef0c7-60e9-41ec-bbbd-cd57c1f2e0c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:49.800029 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-4a3ef0c7-60e9-41ec-bbbd-cd57c1f2e0c8 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:43:49.801359 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-4a3ef0c7-60e9-41ec-bbbd-cd57c1f2e0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:49.891726 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1180/2351] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:43:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:43:49.903504 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dddaa1-bf98-4e0e-bc3e-d6b375803a65 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:49.905096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-81dddaa1-bf98-4e0e-bc3e-d6b375803a65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:49.975077 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dddaa1-bf98-4e0e-bc3e-d6b375803a65 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:49.979083 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d707a7b-824a-46a9-ad49-831d5c1ff9c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:49.979662 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d707a7b-824a-46a9-ad49-831d5c1ff9c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:49.979946 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d707a7b-824a-46a9-ad49-831d5c1ff9c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:49.980227 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d707a7b-824a-46a9-ad49-831d5c1ff9c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:49.992967 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d707a7b-824a-46a9-ad49-831d5c1ff9c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:49.995286 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d707a7b-824a-46a9-ad49-831d5c1ff9c6 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:50.009151 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1172/2352] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:43:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:43:50.377362 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ef0c7-60e9-41ec-bbbd-cd57c1f2e0c8 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:50.379636 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-4a3ef0c7-60e9-41ec-bbbd-cd57c1f2e0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:50.397814 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ef0c7-60e9-41ec-bbbd-cd57c1f2e0c8 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:50.405785 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae3b0bb-f491-4d43-a4b9-1253c6fb0e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:50.407760 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae3b0bb-f491-4d43-a4b9-1253c6fb0e88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:50.408149 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae3b0bb-f491-4d43-a4b9-1253c6fb0e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:50.409046 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae3b0bb-f491-4d43-a4b9-1253c6fb0e88 admin admin] PATH_INFO: `/v3/projects/65b89c7fe92745328d9c55540d20612b` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:50.410581 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae3b0bb-f491-4d43-a4b9-1253c6fb0e88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65b89c7fe92745328d9c55540d20612b)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:50.412817 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae3b0bb-f491-4d43-a4b9-1253c6fb0e88 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:50.488804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-fae3b0bb-f491-4d43-a4b9-1253c6fb0e88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65b89c7fe92745328d9c55540d20612b', 'request_id': 'req-fae3b0bb-f491-4d43-a4b9-1253c6fb0e88'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:50.491991 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:43:50.500265 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-fae3b0bb-f491-4d43-a4b9-1253c6fb0e88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65b89c7fe92745328d9c55540d20612b', 'request_id': 'req-fae3b0bb-f491-4d43-a4b9-1253c6fb0e88'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:50.511939 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-fae3b0bb-f491-4d43-a4b9-1253c6fb0e88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65b89c7fe92745328d9c55540d20612b', 'request_id': 'req-fae3b0bb-f491-4d43-a4b9-1253c6fb0e88'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:50.522915 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-fae3b0bb-f491-4d43-a4b9-1253c6fb0e88 admin admin] The token cache is being invalidate because project 65b89c7fe92745328d9c55540d20612b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:43:50.523204 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-fae3b0bb-f491-4d43-a4b9-1253c6fb0e88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:50.523856 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-fae3b0bb-f491-4d43-a4b9-1253c6fb0e88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:50.525393 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1181/2353] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:43:50 2022] DELETE /identity/v3/projects/65b89c7fe92745328d9c55540d20612b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:43:50.529458 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d707a7b-824a-46a9-ad49-831d5c1ff9c6 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:50.541852 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d707a7b-824a-46a9-ad49-831d5c1ff9c6 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:50.562663 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:43:50.605265 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2cc4db-c72b-4412-8e76-18c1fbf6c655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:50.606186 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2cc4db-c72b-4412-8e76-18c1fbf6c655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:50.606470 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2cc4db-c72b-4412-8e76-18c1fbf6c655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:50.606746 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2cc4db-c72b-4412-8e76-18c1fbf6c655 admin admin] PATH_INFO: `/v3/users/3422598c40864e5cb13f5457bf71ebdb` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:50.607940 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2cc4db-c72b-4412-8e76-18c1fbf6c655 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3422598c40864e5cb13f5457bf71ebdb)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:50.609907 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2cc4db-c72b-4412-8e76-18c1fbf6c655 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:50.660460 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8f2cc4db-c72b-4412-8e76-18c1fbf6c655 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3422598c40864e5cb13f5457bf71ebdb'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:50.661108 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8f2cc4db-c72b-4412-8e76-18c1fbf6c655 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3422598c40864e5cb13f5457bf71ebdb'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:50.661714 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8f2cc4db-c72b-4412-8e76-18c1fbf6c655 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3422598c40864e5cb13f5457bf71ebdb'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:50.674101 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8f2cc4db-c72b-4412-8e76-18c1fbf6c655 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3422598c40864e5cb13f5457bf71ebdb'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:50.682515 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8f2cc4db-c72b-4412-8e76-18c1fbf6c655 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3422598c40864e5cb13f5457bf71ebdb'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:50.697259 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1173/2354] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:43:50 2022] DELETE /identity/v3/users/3422598c40864e5cb13f5457bf71ebdb => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:43:51.181418 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae3b0bb-f491-4d43-a4b9-1253c6fb0e88 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:51.182849 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-fae3b0bb-f491-4d43-a4b9-1253c6fb0e88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:51.204300 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae3b0bb-f491-4d43-a4b9-1253c6fb0e88 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:51.280865 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d95b4a3-876f-4e8a-8b4e-cee30fe66723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:51.281901 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95b4a3-876f-4e8a-8b4e-cee30fe66723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:51.282222 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95b4a3-876f-4e8a-8b4e-cee30fe66723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:51.282496 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95b4a3-876f-4e8a-8b4e-cee30fe66723 admin admin] PATH_INFO: `/v3/projects/477a6028011f44c68183f514c509eabc` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:51.284346 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d95b4a3-876f-4e8a-8b4e-cee30fe66723 admin admin] RBAC: Authorizing `identity:delete_project(project_id=477a6028011f44c68183f514c509eabc)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:51.286333 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d95b4a3-876f-4e8a-8b4e-cee30fe66723 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:51.362956 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-1d95b4a3-876f-4e8a-8b4e-cee30fe66723 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '477a6028011f44c68183f514c509eabc', 'request_id': 'req-1d95b4a3-876f-4e8a-8b4e-cee30fe66723'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:51.373795 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-1d95b4a3-876f-4e8a-8b4e-cee30fe66723 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '477a6028011f44c68183f514c509eabc', 'request_id': 'req-1d95b4a3-876f-4e8a-8b4e-cee30fe66723'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:51.381679 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-1d95b4a3-876f-4e8a-8b4e-cee30fe66723 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '477a6028011f44c68183f514c509eabc', 'request_id': 'req-1d95b4a3-876f-4e8a-8b4e-cee30fe66723'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:51.393549 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-1d95b4a3-876f-4e8a-8b4e-cee30fe66723 admin admin] The token cache is being invalidate because project 477a6028011f44c68183f514c509eabc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:43:51.393831 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-1d95b4a3-876f-4e8a-8b4e-cee30fe66723 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:51.394502 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-1d95b4a3-876f-4e8a-8b4e-cee30fe66723 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:51.397132 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1182/2355] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:43:51 2022] DELETE /identity/v3/projects/477a6028011f44c68183f514c509eabc => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:43:51.402350 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064b855-ae3f-47d0-9696-292392d92dc8 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:51.402643 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064b855-ae3f-47d0-9696-292392d92dc8 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:51.402931 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064b855-ae3f-47d0-9696-292392d92dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:51.424372 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-9064b855-ae3f-47d0-9696-292392d92dc8 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:43:51.425601 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-9064b855-ae3f-47d0-9696-292392d92dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:51.492262 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1174/2356] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:43:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:43:51.501208 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d964a70-5972-4b85-9874-287e2eb861df None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:51.501501 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d964a70-5972-4b85-9874-287e2eb861df None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:51.501836 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d964a70-5972-4b85-9874-287e2eb861df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:51.524162 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-6d964a70-5972-4b85-9874-287e2eb861df None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:43:51.528728 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-6d964a70-5972-4b85-9874-287e2eb861df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:51.551328 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1183/2357] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:43:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:43:51.555908 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9064b855-ae3f-47d0-9696-292392d92dc8 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:51.568067 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9064b855-ae3f-47d0-9696-292392d92dc8 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:51.578665 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35adc73-4535-4978-ba4c-85bd805cc59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:51.580225 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35adc73-4535-4978-ba4c-85bd805cc59b admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:51.580668 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35adc73-4535-4978-ba4c-85bd805cc59b admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:51.580989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35adc73-4535-4978-ba4c-85bd805cc59b admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:51.582069 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35adc73-4535-4978-ba4c-85bd805cc59b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:43:51.582448 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35adc73-4535-4978-ba4c-85bd805cc59b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:51.584523 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35adc73-4535-4978-ba4c-85bd805cc59b admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:51.598174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1175/2358] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:43:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:43:51.604172 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d964a70-5972-4b85-9874-287e2eb861df None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:51.614275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d964a70-5972-4b85-9874-287e2eb861df None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:51.619970 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26116ecf-ed59-42de-b71c-7b9cece82968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:51.620516 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26116ecf-ed59-42de-b71c-7b9cece82968 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:51.620783 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26116ecf-ed59-42de-b71c-7b9cece82968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:51.621054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26116ecf-ed59-42de-b71c-7b9cece82968 admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:51.621753 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26116ecf-ed59-42de-b71c-7b9cece82968 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:51.623698 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26116ecf-ed59-42de-b71c-7b9cece82968 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:51.657795 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1184/2359] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:43:51 2022] POST /identity/v3/projects => generated 393 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:43:51.663219 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35adc73-4535-4978-ba4c-85bd805cc59b admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:51.674459 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35adc73-4535-4978-ba4c-85bd805cc59b admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:51.712112 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:43:51.751036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d74537-deba-4449-9a2b-5afd267abf1f None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:51.751371 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d74537-deba-4449-9a2b-5afd267abf1f None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:51.752099 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d74537-deba-4449-9a2b-5afd267abf1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:51.761958 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1692031b-3a2c-448e-b0c3-f2b65b30323e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:51.762684 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692031b-3a2c-448e-b0c3-f2b65b30323e admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:51.762980 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692031b-3a2c-448e-b0c3-f2b65b30323e admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:51.763259 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692031b-3a2c-448e-b0c3-f2b65b30323e admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:51.764058 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1692031b-3a2c-448e-b0c3-f2b65b30323e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:51.766017 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1692031b-3a2c-448e-b0c3-f2b65b30323e admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:51.776406 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-11d74537-deba-4449-9a2b-5afd267abf1f None None] MFA Rules not processed for user `67aa048d409c46e9a729aec9d1b2286e`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:43:51.778461 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-11d74537-deba-4449-9a2b-5afd267abf1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:51.798947 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1176/2360] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:43:51 2022] POST /identity/v3/users => generated 297 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:43:51.801234 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1692031b-3a2c-448e-b0c3-f2b65b30323e admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:51.815896 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1692031b-3a2c-448e-b0c3-f2b65b30323e admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:51.823743 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a3f52-53c4-4bac-a6de-d84cb75b7fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:51.824333 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a3f52-53c4-4bac-a6de-d84cb75b7fa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:51.824629 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a3f52-53c4-4bac-a6de-d84cb75b7fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:51.825043 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a3f52-53c4-4bac-a6de-d84cb75b7fa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:51.825767 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a3f52-53c4-4bac-a6de-d84cb75b7fa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:51.827554 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a3f52-53c4-4bac-a6de-d84cb75b7fa1 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:51.838288 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:43:51.843348 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1177/2361] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:43:51 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:43:51.846995 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a3f52-53c4-4bac-a6de-d84cb75b7fa1 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:51.859406 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a3f52-53c4-4bac-a6de-d84cb75b7fa1 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:51.863033 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1185/2362] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Jun 15 19:43:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:43:51.865195 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9346850a-ba7a-4de8-a561-6a2f3879215b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:51.866404 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346850a-ba7a-4de8-a561-6a2f3879215b admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:51.866909 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346850a-ba7a-4de8-a561-6a2f3879215b admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:51.867395 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346850a-ba7a-4de8-a561-6a2f3879215b admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:51.868702 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9346850a-ba7a-4de8-a561-6a2f3879215b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:51.871710 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9346850a-ba7a-4de8-a561-6a2f3879215b admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:51.877923 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d74537-deba-4449-9a2b-5afd267abf1f None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:51.880469 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-11d74537-deba-4449-9a2b-5afd267abf1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:51.884794 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1178/2363] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:43:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:43:51.890501 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9346850a-ba7a-4de8-a561-6a2f3879215b admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:51.902700 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9346850a-ba7a-4de8-a561-6a2f3879215b admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:51.910341 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5afddba-d373-4c9b-a8e5-146328d17a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:51.912383 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5afddba-d373-4c9b-a8e5-146328d17a85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:51.913243 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5afddba-d373-4c9b-a8e5-146328d17a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:51.913833 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5afddba-d373-4c9b-a8e5-146328d17a85 admin admin] PATH_INFO: `/v3/projects/d079f314663a43ec8f5a4b95f32cacd3/users/522979e0a77b44dda0b5735f1f8c83c6/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:51.934867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5afddba-d373-4c9b-a8e5-146328d17a85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d079f314663a43ec8f5a4b95f32cacd3, user_id=522979e0a77b44dda0b5735f1f8c83c6, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:51.937841 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5afddba-d373-4c9b-a8e5-146328d17a85 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:51.961841 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1179/2364] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:43:51 2022] PUT /identity/v3/projects/d079f314663a43ec8f5a4b95f32cacd3/users/522979e0a77b44dda0b5735f1f8c83c6/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:43:51.974987 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d74537-deba-4449-9a2b-5afd267abf1f None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:51.977743 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048a465-da67-4aae-b2d0-8f5796b65f4f None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:51.978266 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048a465-da67-4aae-b2d0-8f5796b65f4f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:51.978761 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048a465-da67-4aae-b2d0-8f5796b65f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:52.011199 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-1048a465-da67-4aae-b2d0-8f5796b65f4f None None] MFA Rules not processed for user `522979e0a77b44dda0b5735f1f8c83c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:43:52.012805 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1048a465-da67-4aae-b2d0-8f5796b65f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:52.068227 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db54038b-f91a-494c-9398-ef2dfd00218c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176d5775a70f4524a9b7fe1ce9d426ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:52.069010 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54038b-f91a-494c-9398-ef2dfd00218c service swift] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:52.069346 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54038b-f91a-494c-9398-ef2dfd00218c service swift] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:52.069705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54038b-f91a-494c-9398-ef2dfd00218c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:52.123593 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1180/2365] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:43:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:43:52.124729 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1048a465-da67-4aae-b2d0-8f5796b65f4f None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:52.158762 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db54038b-f91a-494c-9398-ef2dfd00218c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:52.161897 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db54038b-f91a-494c-9398-ef2dfd00218c service swift] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:52.180089 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1186/2366] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Jun 15 19:43:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 19:43:52.227013 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1048a465-da67-4aae-b2d0-8f5796b65f4f None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:52.236240 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee1d44-7cf6-438a-95cd-54604ca22baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:52.237074 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee1d44-7cf6-438a-95cd-54604ca22baa admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:52.237684 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee1d44-7cf6-438a-95cd-54604ca22baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:52.237975 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee1d44-7cf6-438a-95cd-54604ca22baa admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:52.239288 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ee1d44-7cf6-438a-95cd-54604ca22baa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:52.243309 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ee1d44-7cf6-438a-95cd-54604ca22baa admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:52.250667 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:43:52.316199 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1181/2367] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:43:52 2022] POST /identity/v3/projects => generated 393 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:43:52.322434 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db54038b-f91a-494c-9398-ef2dfd00218c service swift] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:52.339861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db54038b-f91a-494c-9398-ef2dfd00218c service swift] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:52.412515 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:43:52.464241 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d77c9-6e68-4ff9-b95c-79bf07a85fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:52.466670 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d77c9-6e68-4ff9-b95c-79bf07a85fc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:52.466956 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d77c9-6e68-4ff9-b95c-79bf07a85fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:52.467246 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d77c9-6e68-4ff9-b95c-79bf07a85fc4 admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:52.468255 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60d77c9-6e68-4ff9-b95c-79bf07a85fc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:52.472319 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60d77c9-6e68-4ff9-b95c-79bf07a85fc4 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:52.511928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1187/2368] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:43:52 2022] POST /identity/v3/users => generated 303 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:43:52.514731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee1d44-7cf6-438a-95cd-54604ca22baa admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:52.528358 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee1d44-7cf6-438a-95cd-54604ca22baa admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:52.543157 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693eb1e7-66dc-4e34-abbd-825779c6ccc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:52.543849 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693eb1e7-66dc-4e34-abbd-825779c6ccc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:52.544133 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693eb1e7-66dc-4e34-abbd-825779c6ccc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:52.544416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693eb1e7-66dc-4e34-abbd-825779c6ccc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:52.545219 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693eb1e7-66dc-4e34-abbd-825779c6ccc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:52.546850 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693eb1e7-66dc-4e34-abbd-825779c6ccc2 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:52.579004 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1182/2369] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:43:52 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:43:52.581489 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d77c9-6e68-4ff9-b95c-79bf07a85fc4 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:52.609771 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d77c9-6e68-4ff9-b95c-79bf07a85fc4 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:52.618745 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a902d67-fa18-4e8c-92aa-2f49e258c9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:52.620106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a902d67-fa18-4e8c-92aa-2f49e258c9ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:52.621450 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a902d67-fa18-4e8c-92aa-2f49e258c9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:52.621450 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a902d67-fa18-4e8c-92aa-2f49e258c9ea admin admin] PATH_INFO: `/v3/projects/fb84f154f7304adcac36bea9ae89ba1a/users/e6b3ec9a1c8d42069f175efb7975f372/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:52.640180 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a902d67-fa18-4e8c-92aa-2f49e258c9ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb84f154f7304adcac36bea9ae89ba1a, user_id=e6b3ec9a1c8d42069f175efb7975f372, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:52.644552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a902d67-fa18-4e8c-92aa-2f49e258c9ea admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:52.668890 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1188/2370] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:43:52 2022] PUT /identity/v3/projects/fb84f154f7304adcac36bea9ae89ba1a/users/e6b3ec9a1c8d42069f175efb7975f372/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:43:52.673237 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693eb1e7-66dc-4e34-abbd-825779c6ccc2 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:52.691637 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693eb1e7-66dc-4e34-abbd-825779c6ccc2 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:52.793216 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74782f21-e53d-4d32-86e6-c5c4c3283ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:52.794112 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74782f21-e53d-4d32-86e6-c5c4c3283ef3 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:52.794395 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74782f21-e53d-4d32-86e6-c5c4c3283ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:52.794738 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74782f21-e53d-4d32-86e6-c5c4c3283ef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:52.795556 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74782f21-e53d-4d32-86e6-c5c4c3283ef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:52.797645 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74782f21-e53d-4d32-86e6-c5c4c3283ef3 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:52.810815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1183/2371] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:43:52 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:43:52.815977 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a902d67-fa18-4e8c-92aa-2f49e258c9ea admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:52.835177 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a902d67-fa18-4e8c-92aa-2f49e258c9ea admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:52.841111 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b13a8-92f4-4312-a704-b9695877a685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:52.842201 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b13a8-92f4-4312-a704-b9695877a685 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:52.842496 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b13a8-92f4-4312-a704-b9695877a685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:52.842840 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b13a8-92f4-4312-a704-b9695877a685 admin admin] PATH_INFO: `/v3/projects/fb84f154f7304adcac36bea9ae89ba1a/users/e6b3ec9a1c8d42069f175efb7975f372/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:52.846681 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97b13a8-92f4-4312-a704-b9695877a685 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb84f154f7304adcac36bea9ae89ba1a, user_id=e6b3ec9a1c8d42069f175efb7975f372, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:52.849741 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97b13a8-92f4-4312-a704-b9695877a685 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:52.862762 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1189/2372] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:43:52 2022] PUT /identity/v3/projects/fb84f154f7304adcac36bea9ae89ba1a/users/e6b3ec9a1c8d42069f175efb7975f372/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:43:52.874742 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ce408-21a4-42aa-8ca6-72afd64123c9 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:52.875690 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ce408-21a4-42aa-8ca6-72afd64123c9 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:52.876232 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ce408-21a4-42aa-8ca6-72afd64123c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:52.921236 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-410ce408-21a4-42aa-8ca6-72afd64123c9 None None] MFA Rules not processed for user `e6b3ec9a1c8d42069f175efb7975f372`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:43:52.922692 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-410ce408-21a4-42aa-8ca6-72afd64123c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:53.032353 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1184/2373] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:43:52 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:43:53.046472 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b13a8-92f4-4312-a704-b9695877a685 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:53.048192 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-c97b13a8-92f4-4312-a704-b9695877a685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:53.131122 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b13a8-92f4-4312-a704-b9695877a685 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:53.136218 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047dff96-4021-48fd-8a68-7ba5ff41dcf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:53.137041 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dff96-4021-48fd-8a68-7ba5ff41dcf1 service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:53.137378 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dff96-4021-48fd-8a68-7ba5ff41dcf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:53.137831 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dff96-4021-48fd-8a68-7ba5ff41dcf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:53.148456 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047dff96-4021-48fd-8a68-7ba5ff41dcf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:53.149909 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047dff96-4021-48fd-8a68-7ba5ff41dcf1 service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:53.165399 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1190/2374] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:43:53 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:43:53.224831 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ce408-21a4-42aa-8ca6-72afd64123c9 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:53.227425 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-410ce408-21a4-42aa-8ca6-72afd64123c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:53.239910 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ce408-21a4-42aa-8ca6-72afd64123c9 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:53.243863 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f3cb0-911d-4ef5-9c52-f1de78485226 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:53.244823 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f3cb0-911d-4ef5-9c52-f1de78485226 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:53.245136 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f3cb0-911d-4ef5-9c52-f1de78485226 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:53.245423 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f3cb0-911d-4ef5-9c52-f1de78485226 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:53.332577 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:43:53.336913 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f3cb0-911d-4ef5-9c52-f1de78485226 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:53.347596 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f3cb0-911d-4ef5-9c52-f1de78485226 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:53.360767 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1185/2375] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:43:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:43:57.040616 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047dff96-4021-48fd-8a68-7ba5ff41dcf1 service cinder] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:57.109271 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047dff96-4021-48fd-8a68-7ba5ff41dcf1 service cinder] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:57.113910 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0799e-3769-4565-a370-5a85c87f9975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:57.114839 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0799e-3769-4565-a370-5a85c87f9975 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:57.115123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0799e-3769-4565-a370-5a85c87f9975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:57.115416 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0799e-3769-4565-a370-5a85c87f9975 admin admin] PATH_INFO: `/v3/users/522979e0a77b44dda0b5735f1f8c83c6` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:57.116661 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de0799e-3769-4565-a370-5a85c87f9975 admin admin] RBAC: Authorizing `identity:delete_user(user_id=522979e0a77b44dda0b5735f1f8c83c6)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:57.118669 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de0799e-3769-4565-a370-5a85c87f9975 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:57.153085 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:43:57.174056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-5de0799e-3769-4565-a370-5a85c87f9975 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '522979e0a77b44dda0b5735f1f8c83c6'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:57.174790 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-5de0799e-3769-4565-a370-5a85c87f9975 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '522979e0a77b44dda0b5735f1f8c83c6'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:57.188392 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-5de0799e-3769-4565-a370-5a85c87f9975 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '522979e0a77b44dda0b5735f1f8c83c6'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:57.189456 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-5de0799e-3769-4565-a370-5a85c87f9975 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '522979e0a77b44dda0b5735f1f8c83c6'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:57.201158 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-5de0799e-3769-4565-a370-5a85c87f9975 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '522979e0a77b44dda0b5735f1f8c83c6'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:57.219552 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1191/2376] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:43:57 2022] DELETE /identity/v3/users/522979e0a77b44dda0b5735f1f8c83c6 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:43:57.230123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9e3a4-bc58-4b97-8539-cc1ca5972fc1 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:57.230123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9e3a4-bc58-4b97-8539-cc1ca5972fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:57.230123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9e3a4-bc58-4b97-8539-cc1ca5972fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:57.274647 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-3cd9e3a4-bc58-4b97-8539-cc1ca5972fc1 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:43:57.276029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3cd9e3a4-bc58-4b97-8539-cc1ca5972fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:57.351104 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1186/2377] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:43:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:43:57.362378 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0799e-3769-4565-a370-5a85c87f9975 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:57.364152 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-5de0799e-3769-4565-a370-5a85c87f9975 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:57.452548 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0799e-3769-4565-a370-5a85c87f9975 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:57.466379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387fbc02-77ef-46d4-bd2b-02f5e5d4d6a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:57.466379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fbc02-77ef-46d4-bd2b-02f5e5d4d6a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:57.466379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fbc02-77ef-46d4-bd2b-02f5e5d4d6a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:57.466379 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fbc02-77ef-46d4-bd2b-02f5e5d4d6a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:57.481044 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387fbc02-77ef-46d4-bd2b-02f5e5d4d6a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:57.482718 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387fbc02-77ef-46d4-bd2b-02f5e5d4d6a3 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:57.505062 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1192/2378] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:43:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:43:57.823303 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9e3a4-bc58-4b97-8539-cc1ca5972fc1 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:57.824904 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3cd9e3a4-bc58-4b97-8539-cc1ca5972fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:57.837894 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9e3a4-bc58-4b97-8539-cc1ca5972fc1 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:57.842754 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce2875-09cb-4ca2-a45d-6b457f5c434d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:57.843734 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce2875-09cb-4ca2-a45d-6b457f5c434d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:57.844088 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce2875-09cb-4ca2-a45d-6b457f5c434d admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:57.844447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce2875-09cb-4ca2-a45d-6b457f5c434d admin admin] PATH_INFO: `/v3/projects/d079f314663a43ec8f5a4b95f32cacd3` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:57.846305 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ce2875-09cb-4ca2-a45d-6b457f5c434d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d079f314663a43ec8f5a4b95f32cacd3)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:57.848504 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ce2875-09cb-4ca2-a45d-6b457f5c434d admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:57.914638 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-37ce2875-09cb-4ca2-a45d-6b457f5c434d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd079f314663a43ec8f5a4b95f32cacd3', 'request_id': 'req-37ce2875-09cb-4ca2-a45d-6b457f5c434d'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:57.925955 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-37ce2875-09cb-4ca2-a45d-6b457f5c434d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd079f314663a43ec8f5a4b95f32cacd3', 'request_id': 'req-37ce2875-09cb-4ca2-a45d-6b457f5c434d'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:57.937925 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-37ce2875-09cb-4ca2-a45d-6b457f5c434d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd079f314663a43ec8f5a4b95f32cacd3', 'request_id': 'req-37ce2875-09cb-4ca2-a45d-6b457f5c434d'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:57.952003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-37ce2875-09cb-4ca2-a45d-6b457f5c434d admin admin] The token cache is being invalidate because project d079f314663a43ec8f5a4b95f32cacd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:43:57.952549 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-37ce2875-09cb-4ca2-a45d-6b457f5c434d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:57.953385 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-37ce2875-09cb-4ca2-a45d-6b457f5c434d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:57.955532 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1187/2379] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:43:57 2022] DELETE /identity/v3/projects/d079f314663a43ec8f5a4b95f32cacd3 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:43:57.959642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387fbc02-77ef-46d4-bd2b-02f5e5d4d6a3 service neutron] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:57.971753 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387fbc02-77ef-46d4-bd2b-02f5e5d4d6a3 service neutron] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:58.049561 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55eb1d-aad3-4c36-80cf-d0d348426fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:58.050687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55eb1d-aad3-4c36-80cf-d0d348426fe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:58.051158 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55eb1d-aad3-4c36-80cf-d0d348426fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:58.051446 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55eb1d-aad3-4c36-80cf-d0d348426fe8 admin admin] PATH_INFO: `/v3/users/e6b3ec9a1c8d42069f175efb7975f372` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:58.052838 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec55eb1d-aad3-4c36-80cf-d0d348426fe8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6b3ec9a1c8d42069f175efb7975f372)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:58.054924 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec55eb1d-aad3-4c36-80cf-d0d348426fe8 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:58.109487 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-ec55eb1d-aad3-4c36-80cf-d0d348426fe8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6b3ec9a1c8d42069f175efb7975f372'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:58.112242 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-ec55eb1d-aad3-4c36-80cf-d0d348426fe8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6b3ec9a1c8d42069f175efb7975f372'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:58.115914 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:43:58.122955 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-ec55eb1d-aad3-4c36-80cf-d0d348426fe8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6b3ec9a1c8d42069f175efb7975f372'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:58.123701 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-ec55eb1d-aad3-4c36-80cf-d0d348426fe8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6b3ec9a1c8d42069f175efb7975f372'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:58.134485 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-ec55eb1d-aad3-4c36-80cf-d0d348426fe8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6b3ec9a1c8d42069f175efb7975f372'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:58.155515 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1193/2380] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:43:57 2022] DELETE /identity/v3/users/e6b3ec9a1c8d42069f175efb7975f372 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:43:58.480660 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce2875-09cb-4ca2-a45d-6b457f5c434d admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:58.482279 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-37ce2875-09cb-4ca2-a45d-6b457f5c434d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:58.497807 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce2875-09cb-4ca2-a45d-6b457f5c434d admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:58.573478 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c434d18e-4efb-46b7-a6bd-b1375e6e172c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:58.574694 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434d18e-4efb-46b7-a6bd-b1375e6e172c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:58.575088 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434d18e-4efb-46b7-a6bd-b1375e6e172c admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:58.575484 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434d18e-4efb-46b7-a6bd-b1375e6e172c admin admin] PATH_INFO: `/v3/projects/fb84f154f7304adcac36bea9ae89ba1a` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:58.576918 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c434d18e-4efb-46b7-a6bd-b1375e6e172c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb84f154f7304adcac36bea9ae89ba1a)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:58.579570 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c434d18e-4efb-46b7-a6bd-b1375e6e172c admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:58.623728 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:43:58.689499 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-c434d18e-4efb-46b7-a6bd-b1375e6e172c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb84f154f7304adcac36bea9ae89ba1a', 'request_id': 'req-c434d18e-4efb-46b7-a6bd-b1375e6e172c'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:58.700503 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-c434d18e-4efb-46b7-a6bd-b1375e6e172c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb84f154f7304adcac36bea9ae89ba1a', 'request_id': 'req-c434d18e-4efb-46b7-a6bd-b1375e6e172c'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:58.717192 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-c434d18e-4efb-46b7-a6bd-b1375e6e172c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb84f154f7304adcac36bea9ae89ba1a', 'request_id': 'req-c434d18e-4efb-46b7-a6bd-b1375e6e172c'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:58.730026 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-c434d18e-4efb-46b7-a6bd-b1375e6e172c admin admin] The token cache is being invalidate because project fb84f154f7304adcac36bea9ae89ba1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:43:58.730554 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-c434d18e-4efb-46b7-a6bd-b1375e6e172c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:58.731315 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-c434d18e-4efb-46b7-a6bd-b1375e6e172c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:43:58.732931 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1188/2381] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:43:58 2022] DELETE /identity/v3/projects/fb84f154f7304adcac36bea9ae89ba1a => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:43:58.738914 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a1cc4-6c61-4d2f-97a1-f735a6a2ad1e None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:58.739241 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a1cc4-6c61-4d2f-97a1-f735a6a2ad1e None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:58.739521 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a1cc4-6c61-4d2f-97a1-f735a6a2ad1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:58.760081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-ed1a1cc4-6c61-4d2f-97a1-f735a6a2ad1e None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:43:58.761400 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-ed1a1cc4-6c61-4d2f-97a1-f735a6a2ad1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:58.825807 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1194/2382] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:43:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:43:58.833613 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e13ec-df09-4d27-b95d-fabd428d0cd0 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:58.834054 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e13ec-df09-4d27-b95d-fabd428d0cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:58.834295 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e13ec-df09-4d27-b95d-fabd428d0cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:58.860266 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-339e13ec-df09-4d27-b95d-fabd428d0cd0 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:43:58.861989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-339e13ec-df09-4d27-b95d-fabd428d0cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:58.871273 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1189/2383] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:43:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:43:58.875761 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a1cc4-6c61-4d2f-97a1-f735a6a2ad1e None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:58.886305 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a1cc4-6c61-4d2f-97a1-f735a6a2ad1e None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:58.891236 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49738def-5695-4421-8dcf-dadb6d0a0022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:58.891955 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49738def-5695-4421-8dcf-dadb6d0a0022 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:58.892241 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49738def-5695-4421-8dcf-dadb6d0a0022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:58.892525 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49738def-5695-4421-8dcf-dadb6d0a0022 admin admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:58.893267 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49738def-5695-4421-8dcf-dadb6d0a0022 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:43:58.893571 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49738def-5695-4421-8dcf-dadb6d0a0022 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:58.895228 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49738def-5695-4421-8dcf-dadb6d0a0022 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:58.908551 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1195/2384] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:43:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:43:58.913277 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e13ec-df09-4d27-b95d-fabd428d0cd0 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:58.924135 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e13ec-df09-4d27-b95d-fabd428d0cd0 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:58.929142 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96657d47-08e5-48d6-a1e0-782cf48bc63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:58.931682 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96657d47-08e5-48d6-a1e0-782cf48bc63e admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:58.931682 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96657d47-08e5-48d6-a1e0-782cf48bc63e admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:58.931682 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96657d47-08e5-48d6-a1e0-782cf48bc63e admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:58.931682 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96657d47-08e5-48d6-a1e0-782cf48bc63e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:58.933038 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96657d47-08e5-48d6-a1e0-782cf48bc63e admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:58.963986 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1190/2385] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:43:58 2022] POST /identity/v3/projects => generated 409 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:43:58.968374 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49738def-5695-4421-8dcf-dadb6d0a0022 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:58.978579 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49738def-5695-4421-8dcf-dadb6d0a0022 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:59.056433 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d36efda-0a61-43ba-8c93-737f4d2f64fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:59.057120 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36efda-0a61-43ba-8c93-737f4d2f64fd admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:59.057409 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36efda-0a61-43ba-8c93-737f4d2f64fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:59.057683 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36efda-0a61-43ba-8c93-737f4d2f64fd admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:59.058417 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d36efda-0a61-43ba-8c93-737f4d2f64fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:59.060368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d36efda-0a61-43ba-8c93-737f4d2f64fd admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:59.086036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1196/2386] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:43:58 2022] POST /identity/v3/users => generated 305 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:43:59.086342 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96657d47-08e5-48d6-a1e0-782cf48bc63e admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:59.096842 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96657d47-08e5-48d6-a1e0-782cf48bc63e admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:59.101825 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e626ee4d-98ba-4995-a53c-fa9dd009fba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:59.102541 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626ee4d-98ba-4995-a53c-fa9dd009fba2 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:59.102836 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626ee4d-98ba-4995-a53c-fa9dd009fba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:59.103105 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626ee4d-98ba-4995-a53c-fa9dd009fba2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:59.103847 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e626ee4d-98ba-4995-a53c-fa9dd009fba2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:59.105473 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e626ee4d-98ba-4995-a53c-fa9dd009fba2 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:59.121516 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1191/2387] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:43:59.122458 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d36efda-0a61-43ba-8c93-737f4d2f64fd admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:59.134028 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d36efda-0a61-43ba-8c93-737f4d2f64fd admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:59.139277 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab97bb-de2a-40c0-b1f5-fc568499dcbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:59.139961 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab97bb-de2a-40c0-b1f5-fc568499dcbf admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:59.140250 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab97bb-de2a-40c0-b1f5-fc568499dcbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:59.140528 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab97bb-de2a-40c0-b1f5-fc568499dcbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:59.141275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab97bb-de2a-40c0-b1f5-fc568499dcbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:59.142986 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab97bb-de2a-40c0-b1f5-fc568499dcbf admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:59.153992 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1197/2388] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:43:59.159004 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e626ee4d-98ba-4995-a53c-fa9dd009fba2 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:59.169577 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e626ee4d-98ba-4995-a53c-fa9dd009fba2 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:59.175536 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e1a0e-4415-4d1c-af35-481ccf5d2a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:59.176460 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e1a0e-4415-4d1c-af35-481ccf5d2a0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:59.176747 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e1a0e-4415-4d1c-af35-481ccf5d2a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:59.177025 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e1a0e-4415-4d1c-af35-481ccf5d2a0f admin admin] PATH_INFO: `/v3/projects/4003fc5ac6934f0b95e129a1ef865eed/users/391ab0a296654627a6da946f37ba4f86/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:59.195453 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e1a0e-4415-4d1c-af35-481ccf5d2a0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4003fc5ac6934f0b95e129a1ef865eed, user_id=391ab0a296654627a6da946f37ba4f86, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:59.198557 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e1a0e-4415-4d1c-af35-481ccf5d2a0f admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:59.214600 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1192/2389] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:43:59 2022] PUT /identity/v3/projects/4003fc5ac6934f0b95e129a1ef865eed/users/391ab0a296654627a6da946f37ba4f86/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:43:59.225836 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef555f-ef0c-4707-95c4-a04d9a99a9cc None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:59.226384 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef555f-ef0c-4707-95c4-a04d9a99a9cc None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:59.226741 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef555f-ef0c-4707-95c4-a04d9a99a9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:59.263583 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-65ef555f-ef0c-4707-95c4-a04d9a99a9cc None None] MFA Rules not processed for user `391ab0a296654627a6da946f37ba4f86`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:43:59.264853 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-65ef555f-ef0c-4707-95c4-a04d9a99a9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:43:59.294613 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:43:59.370131 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1198/2390] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:43:59 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:43:59.373551 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e1a0e-4415-4d1c-af35-481ccf5d2a0f admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:59.450389 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e1a0e-4415-4d1c-af35-481ccf5d2a0f admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:59.456092 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2448e-0806-4c9b-b90b-5a6f9f417ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:59.456847 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2448e-0806-4c9b-b90b-5a6f9f417ce1 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:59.457200 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2448e-0806-4c9b-b90b-5a6f9f417ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:59.457482 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2448e-0806-4c9b-b90b-5a6f9f417ce1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:59.458247 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c2448e-0806-4c9b-b90b-5a6f9f417ce1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:59.460423 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c2448e-0806-4c9b-b90b-5a6f9f417ce1 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:59.501659 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1193/2391] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:43:59 2022] POST /identity/v3/projects => generated 409 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:43:59.505922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef555f-ef0c-4707-95c4-a04d9a99a9cc None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:59.525716 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef555f-ef0c-4707-95c4-a04d9a99a9cc None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:59.607022 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3576a-5d88-4566-92be-6ad9253173f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:59.607823 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3576a-5d88-4566-92be-6ad9253173f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:59.608185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3576a-5d88-4566-92be-6ad9253173f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:59.608537 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3576a-5d88-4566-92be-6ad9253173f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:59.609337 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e3576a-5d88-4566-92be-6ad9253173f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:59.611488 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e3576a-5d88-4566-92be-6ad9253173f9 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:59.638915 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1199/2392] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:43:59 2022] POST /identity/v3/users => generated 311 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:43:59.643354 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2448e-0806-4c9b-b90b-5a6f9f417ce1 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:59.656685 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2448e-0806-4c9b-b90b-5a6f9f417ce1 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:59.665303 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a259f2b-5fac-44a0-ab30-b4028fd0c1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:59.666361 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a259f2b-5fac-44a0-ab30-b4028fd0c1fb admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:59.666653 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a259f2b-5fac-44a0-ab30-b4028fd0c1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:59.666935 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a259f2b-5fac-44a0-ab30-b4028fd0c1fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:59.667719 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a259f2b-5fac-44a0-ab30-b4028fd0c1fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:59.669394 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a259f2b-5fac-44a0-ab30-b4028fd0c1fb admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:59.683350 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1194/2393] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:43:59.689664 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3576a-5d88-4566-92be-6ad9253173f9 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:59.700380 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3576a-5d88-4566-92be-6ad9253173f9 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:59.706593 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a14b7-16f3-4bd1-89ac-cdb1f3d1e395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:59.708000 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a14b7-16f3-4bd1-89ac-cdb1f3d1e395 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:59.708498 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a14b7-16f3-4bd1-89ac-cdb1f3d1e395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:59.708971 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a14b7-16f3-4bd1-89ac-cdb1f3d1e395 admin admin] PATH_INFO: `/v3/projects/dc922d874d844a36a624bc8bccaed063/users/463a176cc1374d1cbd856332fc18a432/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:59.728365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a14b7-16f3-4bd1-89ac-cdb1f3d1e395 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc922d874d844a36a624bc8bccaed063, user_id=463a176cc1374d1cbd856332fc18a432, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:59.731118 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a14b7-16f3-4bd1-89ac-cdb1f3d1e395 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:59.746704 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1200/2394] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:43:59 2022] PUT /identity/v3/projects/dc922d874d844a36a624bc8bccaed063/users/463a176cc1374d1cbd856332fc18a432/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:43:59.750802 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a259f2b-5fac-44a0-ab30-b4028fd0c1fb admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:59.773862 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a259f2b-5fac-44a0-ab30-b4028fd0c1fb admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:59.786748 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:43:59.849239 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f188c857-26c5-4026-9caa-85e64f521aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:59.849943 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188c857-26c5-4026-9caa-85e64f521aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:59.850267 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188c857-26c5-4026-9caa-85e64f521aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:59.850548 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188c857-26c5-4026-9caa-85e64f521aa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:59.852202 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f188c857-26c5-4026-9caa-85e64f521aa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:59.854060 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f188c857-26c5-4026-9caa-85e64f521aa1 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:59.872762 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1195/2395] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:43:59.876973 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a14b7-16f3-4bd1-89ac-cdb1f3d1e395 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:43:59.888190 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a14b7-16f3-4bd1-89ac-cdb1f3d1e395 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:43:59.894536 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c4e21-2e23-4626-8438-07815a983e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:43:59.895477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c4e21-2e23-4626-8438-07815a983e4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:59.895762 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c4e21-2e23-4626-8438-07815a983e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:59.896324 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c4e21-2e23-4626-8438-07815a983e4e admin admin] PATH_INFO: `/v3/projects/dc922d874d844a36a624bc8bccaed063/users/463a176cc1374d1cbd856332fc18a432/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:59.898388 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3c4e21-2e23-4626-8438-07815a983e4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc922d874d844a36a624bc8bccaed063, user_id=463a176cc1374d1cbd856332fc18a432, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:43:59.901071 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3c4e21-2e23-4626-8438-07815a983e4e admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:43:59.914847 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1201/2396] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:43:59 2022] PUT /identity/v3/projects/dc922d874d844a36a624bc8bccaed063/users/463a176cc1374d1cbd856332fc18a432/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:43:59.926159 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912480f8-601f-4144-9195-b24ead6fe440 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:43:59.926464 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912480f8-601f-4144-9195-b24ead6fe440 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:43:59.926759 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912480f8-601f-4144-9195-b24ead6fe440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:43:59.967318 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-912480f8-601f-4144-9195-b24ead6fe440 None None] MFA Rules not processed for user `463a176cc1374d1cbd856332fc18a432`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:43:59.968657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-912480f8-601f-4144-9195-b24ead6fe440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:00.066156 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1196/2397] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:43:59 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:44:00.083225 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c4e21-2e23-4626-8438-07815a983e4e admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:00.086601 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-cf3c4e21-2e23-4626-8438-07815a983e4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:00.167645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c4e21-2e23-4626-8438-07815a983e4e admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:00.172120 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b93b1a-04ec-4bdf-8531-b94273d37110 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:00.172859 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b93b1a-04ec-4bdf-8531-b94273d37110 service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:00.173293 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b93b1a-04ec-4bdf-8531-b94273d37110 service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:00.173667 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b93b1a-04ec-4bdf-8531-b94273d37110 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:00.184669 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b93b1a-04ec-4bdf-8531-b94273d37110 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:00.186362 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b93b1a-04ec-4bdf-8531-b94273d37110 service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:00.198444 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1202/2398] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:44:00 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:44:00.411332 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912480f8-601f-4144-9195-b24ead6fe440 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:00.434045 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912480f8-601f-4144-9195-b24ead6fe440 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:00.437921 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55181919-3191-402e-825a-7cbe90f7b643 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:00.438701 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55181919-3191-402e-825a-7cbe90f7b643 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:00.442549 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55181919-3191-402e-825a-7cbe90f7b643 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:00.442968 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55181919-3191-402e-825a-7cbe90f7b643 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:00.508999 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55181919-3191-402e-825a-7cbe90f7b643 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:00.510515 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55181919-3191-402e-825a-7cbe90f7b643 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:00.520168 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1197/2399] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:44:00 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:44:07.914396 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b93b1a-04ec-4bdf-8531-b94273d37110 service cinder] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:07.937938 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:44:07.989675 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b93b1a-04ec-4bdf-8531-b94273d37110 service cinder] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:07.994834 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5971ef-b3f4-483f-821c-7ff1a3e641e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:07.995823 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5971ef-b3f4-483f-821c-7ff1a3e641e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:07.996140 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5971ef-b3f4-483f-821c-7ff1a3e641e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:07.996541 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5971ef-b3f4-483f-821c-7ff1a3e641e5 admin admin] PATH_INFO: `/v3/users/391ab0a296654627a6da946f37ba4f86` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:07.997779 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5971ef-b3f4-483f-821c-7ff1a3e641e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=391ab0a296654627a6da946f37ba4f86)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:07.999738 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5971ef-b3f4-483f-821c-7ff1a3e641e5 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:08.063973 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-fe5971ef-b3f4-483f-821c-7ff1a3e641e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '391ab0a296654627a6da946f37ba4f86'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:08.064637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-fe5971ef-b3f4-483f-821c-7ff1a3e641e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '391ab0a296654627a6da946f37ba4f86'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:08.072538 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-fe5971ef-b3f4-483f-821c-7ff1a3e641e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '391ab0a296654627a6da946f37ba4f86'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:08.073185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-fe5971ef-b3f4-483f-821c-7ff1a3e641e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '391ab0a296654627a6da946f37ba4f86'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:08.083631 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-fe5971ef-b3f4-483f-821c-7ff1a3e641e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '391ab0a296654627a6da946f37ba4f86'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:08.109402 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1203/2400] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:44:07 2022] DELETE /identity/v3/users/391ab0a296654627a6da946f37ba4f86 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:44:08.114328 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94b366-7078-4165-8939-79d1ea9654e3 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:08.114847 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94b366-7078-4165-8939-79d1ea9654e3 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:08.114943 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94b366-7078-4165-8939-79d1ea9654e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:08.135257 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-cf94b366-7078-4165-8939-79d1ea9654e3 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:44:08.136836 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-cf94b366-7078-4165-8939-79d1ea9654e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:08.183428 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:44:08.267365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1198/2401] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:44:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:44:08.267469 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5971ef-b3f4-483f-821c-7ff1a3e641e5 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:08.267469 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-fe5971ef-b3f4-483f-821c-7ff1a3e641e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:08.343416 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5971ef-b3f4-483f-821c-7ff1a3e641e5 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:08.348581 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2d6f9-c2ea-4e18-b4b6-53db212b8dd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:08.349292 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2d6f9-c2ea-4e18-b4b6-53db212b8dd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:08.349657 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2d6f9-c2ea-4e18-b4b6-53db212b8dd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:08.349862 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2d6f9-c2ea-4e18-b4b6-53db212b8dd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:08.366779 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab2d6f9-c2ea-4e18-b4b6-53db212b8dd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:08.369831 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab2d6f9-c2ea-4e18-b4b6-53db212b8dd9 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:08.387291 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1204/2402] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:44:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:44:08.726500 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf94b366-7078-4165-8939-79d1ea9654e3 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:08.728357 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-cf94b366-7078-4165-8939-79d1ea9654e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:08.747705 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf94b366-7078-4165-8939-79d1ea9654e3 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:08.758210 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea7ef24-2bb5-42d4-a451-a0b4fb773849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:08.759606 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7ef24-2bb5-42d4-a451-a0b4fb773849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:08.760056 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7ef24-2bb5-42d4-a451-a0b4fb773849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:08.760529 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7ef24-2bb5-42d4-a451-a0b4fb773849 admin admin] PATH_INFO: `/v3/projects/4003fc5ac6934f0b95e129a1ef865eed` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:08.762934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea7ef24-2bb5-42d4-a451-a0b4fb773849 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4003fc5ac6934f0b95e129a1ef865eed)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:08.767727 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea7ef24-2bb5-42d4-a451-a0b4fb773849 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:08.846716 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2ea7ef24-2bb5-42d4-a451-a0b4fb773849 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4003fc5ac6934f0b95e129a1ef865eed', 'request_id': 'req-2ea7ef24-2bb5-42d4-a451-a0b4fb773849'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:08.855950 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2ea7ef24-2bb5-42d4-a451-a0b4fb773849 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4003fc5ac6934f0b95e129a1ef865eed', 'request_id': 'req-2ea7ef24-2bb5-42d4-a451-a0b4fb773849'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:08.865716 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2ea7ef24-2bb5-42d4-a451-a0b4fb773849 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4003fc5ac6934f0b95e129a1ef865eed', 'request_id': 'req-2ea7ef24-2bb5-42d4-a451-a0b4fb773849'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:08.878134 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2ea7ef24-2bb5-42d4-a451-a0b4fb773849 admin admin] The token cache is being invalidate because project 4003fc5ac6934f0b95e129a1ef865eed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:44:08.878422 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2ea7ef24-2bb5-42d4-a451-a0b4fb773849 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:08.879146 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2ea7ef24-2bb5-42d4-a451-a0b4fb773849 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:08.881192 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1199/2403] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:44:08 2022] DELETE /identity/v3/projects/4003fc5ac6934f0b95e129a1ef865eed => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:44:08.885130 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2d6f9-c2ea-4e18-b4b6-53db212b8dd9 service neutron] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:08.899287 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2d6f9-c2ea-4e18-b4b6-53db212b8dd9 service neutron] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:08.939608 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:44:08.961995 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375574a5-639f-42aa-bba5-6b2926398895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:08.962901 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375574a5-639f-42aa-bba5-6b2926398895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:08.963187 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375574a5-639f-42aa-bba5-6b2926398895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:08.963462 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375574a5-639f-42aa-bba5-6b2926398895 admin admin] PATH_INFO: `/v3/users/463a176cc1374d1cbd856332fc18a432` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:08.964673 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375574a5-639f-42aa-bba5-6b2926398895 admin admin] RBAC: Authorizing `identity:delete_user(user_id=463a176cc1374d1cbd856332fc18a432)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:08.966634 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375574a5-639f-42aa-bba5-6b2926398895 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:09.016145 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-375574a5-639f-42aa-bba5-6b2926398895 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '463a176cc1374d1cbd856332fc18a432'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:09.017407 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-375574a5-639f-42aa-bba5-6b2926398895 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '463a176cc1374d1cbd856332fc18a432'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:09.026705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-375574a5-639f-42aa-bba5-6b2926398895 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '463a176cc1374d1cbd856332fc18a432'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:09.027493 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-375574a5-639f-42aa-bba5-6b2926398895 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '463a176cc1374d1cbd856332fc18a432'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:09.036368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-375574a5-639f-42aa-bba5-6b2926398895 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '463a176cc1374d1cbd856332fc18a432'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:09.056572 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1205/2404] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:44:08 2022] DELETE /identity/v3/users/463a176cc1374d1cbd856332fc18a432 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:44:09.367631 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea7ef24-2bb5-42d4-a451-a0b4fb773849 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:09.370120 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-2ea7ef24-2bb5-42d4-a451-a0b4fb773849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:09.387458 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea7ef24-2bb5-42d4-a451-a0b4fb773849 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:09.481129 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cdd5e2-9433-45b7-8288-bd4306ebca7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:09.482058 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cdd5e2-9433-45b7-8288-bd4306ebca7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:09.482404 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cdd5e2-9433-45b7-8288-bd4306ebca7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:09.482708 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cdd5e2-9433-45b7-8288-bd4306ebca7a admin admin] PATH_INFO: `/v3/projects/dc922d874d844a36a624bc8bccaed063` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:09.483982 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cdd5e2-9433-45b7-8288-bd4306ebca7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc922d874d844a36a624bc8bccaed063)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:09.486490 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cdd5e2-9433-45b7-8288-bd4306ebca7a admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:09.540610 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-86cdd5e2-9433-45b7-8288-bd4306ebca7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc922d874d844a36a624bc8bccaed063', 'request_id': 'req-86cdd5e2-9433-45b7-8288-bd4306ebca7a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:09.549587 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-86cdd5e2-9433-45b7-8288-bd4306ebca7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc922d874d844a36a624bc8bccaed063', 'request_id': 'req-86cdd5e2-9433-45b7-8288-bd4306ebca7a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:09.551021 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:44:09.558780 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-86cdd5e2-9433-45b7-8288-bd4306ebca7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc922d874d844a36a624bc8bccaed063', 'request_id': 'req-86cdd5e2-9433-45b7-8288-bd4306ebca7a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:09.571022 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-86cdd5e2-9433-45b7-8288-bd4306ebca7a admin admin] The token cache is being invalidate because project dc922d874d844a36a624bc8bccaed063 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:44:09.571022 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-86cdd5e2-9433-45b7-8288-bd4306ebca7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:09.571965 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-86cdd5e2-9433-45b7-8288-bd4306ebca7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:09.573258 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1200/2405] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:44:09 2022] DELETE /identity/v3/projects/dc922d874d844a36a624bc8bccaed063 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:44:09.579952 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b79b01-df38-4124-b99c-8ab0d34a5b8f None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:09.580239 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b79b01-df38-4124-b99c-8ab0d34a5b8f None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:09.580515 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b79b01-df38-4124-b99c-8ab0d34a5b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:09.598810 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-85b79b01-df38-4124-b99c-8ab0d34a5b8f None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:44:09.600042 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-85b79b01-df38-4124-b99c-8ab0d34a5b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:09.667092 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1206/2406] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:44:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:44:09.676074 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586db04-333b-4d44-a807-d494edb45817 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:09.676294 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586db04-333b-4d44-a807-d494edb45817 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:09.676535 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586db04-333b-4d44-a807-d494edb45817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:09.695505 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-e586db04-333b-4d44-a807-d494edb45817 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:44:09.696867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e586db04-333b-4d44-a807-d494edb45817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:09.706262 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1201/2407] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:44:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:44:09.715849 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b79b01-df38-4124-b99c-8ab0d34a5b8f None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:09.726930 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b79b01-df38-4124-b99c-8ab0d34a5b8f None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:09.731520 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c27cd-3fb3-42c9-b10a-cf23ea862dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:09.732177 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c27cd-3fb3-42c9-b10a-cf23ea862dbf admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:09.732461 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c27cd-3fb3-42c9-b10a-cf23ea862dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:09.732739 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c27cd-3fb3-42c9-b10a-cf23ea862dbf admin admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:09.733466 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c27cd-3fb3-42c9-b10a-cf23ea862dbf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:44:09.733774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c27cd-3fb3-42c9-b10a-cf23ea862dbf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:09.736087 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c27cd-3fb3-42c9-b10a-cf23ea862dbf admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:09.747863 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1207/2408] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:44:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:44:09.752244 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e586db04-333b-4d44-a807-d494edb45817 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:09.762106 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e586db04-333b-4d44-a807-d494edb45817 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:09.767114 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d34bbd-ffa5-48a7-8fee-46600168e59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:09.767839 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d34bbd-ffa5-48a7-8fee-46600168e59b admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:09.768132 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d34bbd-ffa5-48a7-8fee-46600168e59b admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:09.768400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d34bbd-ffa5-48a7-8fee-46600168e59b admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:09.769121 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d34bbd-ffa5-48a7-8fee-46600168e59b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:09.771103 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d34bbd-ffa5-48a7-8fee-46600168e59b admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:09.806236 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1202/2409] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:44:09 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:44:09.810611 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c27cd-3fb3-42c9-b10a-cf23ea862dbf admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:09.821157 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c27cd-3fb3-42c9-b10a-cf23ea862dbf admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:09.869493 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:44:09.886842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346118c1-6099-4200-8c15-c57987a26f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:09.887510 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346118c1-6099-4200-8c15-c57987a26f0b admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:09.887791 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346118c1-6099-4200-8c15-c57987a26f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:09.888074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346118c1-6099-4200-8c15-c57987a26f0b admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:09.888956 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346118c1-6099-4200-8c15-c57987a26f0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:09.890901 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346118c1-6099-4200-8c15-c57987a26f0b admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:09.915783 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1208/2410] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:44:09 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:44:09.919911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d34bbd-ffa5-48a7-8fee-46600168e59b admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:09.931707 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d34bbd-ffa5-48a7-8fee-46600168e59b admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:09.936792 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26727c0-ce64-4782-bed3-24d3e0c30949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:09.937488 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26727c0-ce64-4782-bed3-24d3e0c30949 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:09.937829 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26727c0-ce64-4782-bed3-24d3e0c30949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:09.938221 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26727c0-ce64-4782-bed3-24d3e0c30949 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:09.938973 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26727c0-ce64-4782-bed3-24d3e0c30949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:09.940599 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26727c0-ce64-4782-bed3-24d3e0c30949 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:09.952359 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1203/2411] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:44:09.956405 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346118c1-6099-4200-8c15-c57987a26f0b admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:09.966453 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346118c1-6099-4200-8c15-c57987a26f0b admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:09.971565 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94531aa9-4241-4e4c-8501-d2f2c1533a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:09.972216 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94531aa9-4241-4e4c-8501-d2f2c1533a18 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:09.972498 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94531aa9-4241-4e4c-8501-d2f2c1533a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:09.972771 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94531aa9-4241-4e4c-8501-d2f2c1533a18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:09.973499 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94531aa9-4241-4e4c-8501-d2f2c1533a18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:09.975111 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94531aa9-4241-4e4c-8501-d2f2c1533a18 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:09.984812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1209/2412] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:44:09.990590 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26727c0-ce64-4782-bed3-24d3e0c30949 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:10.002024 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26727c0-ce64-4782-bed3-24d3e0c30949 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:10.009750 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d818898-6261-4525-8ab7-afdad8fa208d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:10.010777 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d818898-6261-4525-8ab7-afdad8fa208d admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:10.011101 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d818898-6261-4525-8ab7-afdad8fa208d admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:10.011522 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d818898-6261-4525-8ab7-afdad8fa208d admin admin] PATH_INFO: `/v3/projects/feaa96a8901e42c18c270c092a40fc83/users/4ca9a3e7826b4928863bd83a9496205a/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:10.027819 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d818898-6261-4525-8ab7-afdad8fa208d admin admin] RBAC: Authorizing `identity:create_grant(project_id=feaa96a8901e42c18c270c092a40fc83, user_id=4ca9a3e7826b4928863bd83a9496205a, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:10.030539 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d818898-6261-4525-8ab7-afdad8fa208d admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:10.045450 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1204/2413] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:44:09 2022] PUT /identity/v3/projects/feaa96a8901e42c18c270c092a40fc83/users/4ca9a3e7826b4928863bd83a9496205a/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:44:10.051094 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a811fa-d087-4bbd-9dcf-d69961805cdc None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:10.051382 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a811fa-d087-4bbd-9dcf-d69961805cdc None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:10.051657 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a811fa-d087-4bbd-9dcf-d69961805cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:10.070313 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-a4a811fa-d087-4bbd-9dcf-d69961805cdc None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:44:10.071431 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-a4a811fa-d087-4bbd-9dcf-d69961805cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:10.135540 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1210/2414] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:44:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:44:10.146967 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d818898-6261-4525-8ab7-afdad8fa208d admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:10.149239 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-4d818898-6261-4525-8ab7-afdad8fa208d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:10.212594 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d818898-6261-4525-8ab7-afdad8fa208d admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:10.216300 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4472a8-3e9f-40f7-8fe7-ff0f62727993 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:10.216946 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4472a8-3e9f-40f7-8fe7-ff0f62727993 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:10.217230 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4472a8-3e9f-40f7-8fe7-ff0f62727993 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:10.217506 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4472a8-3e9f-40f7-8fe7-ff0f62727993 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:10.228130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4472a8-3e9f-40f7-8fe7-ff0f62727993 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:10.229578 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4472a8-3e9f-40f7-8fe7-ff0f62727993 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:10.240911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1205/2415] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:44:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:44:10.469490 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b1bc6-0f73-42c2-8cd4-c239e72b2bee None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:10.469807 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b1bc6-0f73-42c2-8cd4-c239e72b2bee None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:10.470111 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b1bc6-0f73-42c2-8cd4-c239e72b2bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:10.496412 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-fa9b1bc6-0f73-42c2-8cd4-c239e72b2bee None None] MFA Rules not processed for user `67aa048d409c46e9a729aec9d1b2286e`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:44:10.497974 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-fa9b1bc6-0f73-42c2-8cd4-c239e72b2bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:10.615614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1211/2416] 10.222.0.15 () {58 vars in 967 bytes} [Wed Jun 15 19:44:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:44:10.631620 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4472a8-3e9f-40f7-8fe7-ff0f62727993 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:10.633505 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1d4472a8-3e9f-40f7-8fe7-ff0f62727993 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:10.731342 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4472a8-3e9f-40f7-8fe7-ff0f62727993 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:10.737516 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24731195-18cc-4753-a5f7-7db2e99af880 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176d5775a70f4524a9b7fe1ce9d426ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:10.738474 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24731195-18cc-4753-a5f7-7db2e99af880 service swift] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:10.738903 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24731195-18cc-4753-a5f7-7db2e99af880 service swift] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:10.739316 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24731195-18cc-4753-a5f7-7db2e99af880 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:10.753712 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24731195-18cc-4753-a5f7-7db2e99af880 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:10.756026 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24731195-18cc-4753-a5f7-7db2e99af880 service swift] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:10.764438 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1206/2417] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Jun 15 19:44:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 19:44:14.574431 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade6b6a-4fe0-4868-969e-5a4bfc4550bd None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:14.574947 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade6b6a-4fe0-4868-969e-5a4bfc4550bd None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:14.574947 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade6b6a-4fe0-4868-969e-5a4bfc4550bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:14.603004 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-9ade6b6a-4fe0-4868-969e-5a4bfc4550bd None None] MFA Rules not processed for user `4ca9a3e7826b4928863bd83a9496205a`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:44:14.604224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-9ade6b6a-4fe0-4868-969e-5a4bfc4550bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:14.675246 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:44:14.679709 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1212/2418] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:44:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:44:14.684389 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24731195-18cc-4753-a5f7-7db2e99af880 service swift] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:14.695044 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24731195-18cc-4753-a5f7-7db2e99af880 service swift] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:14.699740 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc1a53b-418a-4846-a6e4-62477037bb6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:14.700433 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1a53b-418a-4846-a6e4-62477037bb6a admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:14.700710 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1a53b-418a-4846-a6e4-62477037bb6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:14.700988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1a53b-418a-4846-a6e4-62477037bb6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:14.701698 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc1a53b-418a-4846-a6e4-62477037bb6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:14.704163 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc1a53b-418a-4846-a6e4-62477037bb6a admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:14.714934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:44:14.742207 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1207/2419] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:44:14 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:44:14.744538 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade6b6a-4fe0-4868-969e-5a4bfc4550bd None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:14.757222 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade6b6a-4fe0-4868-969e-5a4bfc4550bd None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:14.832650 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620557d4-478f-4af2-b483-c7491a3371d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:14.833568 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620557d4-478f-4af2-b483-c7491a3371d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:14.834010 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620557d4-478f-4af2-b483-c7491a3371d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:14.834510 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620557d4-478f-4af2-b483-c7491a3371d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:14.835484 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620557d4-478f-4af2-b483-c7491a3371d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:14.837762 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620557d4-478f-4af2-b483-c7491a3371d6 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:14.879976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1213/2420] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:44:14 2022] POST /identity/v3/users => generated 292 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:44:14.883595 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc1a53b-418a-4846-a6e4-62477037bb6a admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:14.895276 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc1a53b-418a-4846-a6e4-62477037bb6a admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:14.904009 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54459e-5fa8-4b4f-91ed-f2ef8b436e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:14.905303 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54459e-5fa8-4b4f-91ed-f2ef8b436e89 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:14.906004 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54459e-5fa8-4b4f-91ed-f2ef8b436e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:14.906504 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54459e-5fa8-4b4f-91ed-f2ef8b436e89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:14.908755 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d54459e-5fa8-4b4f-91ed-f2ef8b436e89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:14.910596 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d54459e-5fa8-4b4f-91ed-f2ef8b436e89 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:14.931765 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1208/2421] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:44:14 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:44:14.941859 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620557d4-478f-4af2-b483-c7491a3371d6 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:14.959535 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620557d4-478f-4af2-b483-c7491a3371d6 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:14.964454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf52ff-8717-40a9-967d-cab8f839c460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:14.965355 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf52ff-8717-40a9-967d-cab8f839c460 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:14.965641 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf52ff-8717-40a9-967d-cab8f839c460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:14.965919 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf52ff-8717-40a9-967d-cab8f839c460 admin admin] PATH_INFO: `/v3/projects/2f7874ef29f7408388914096a7b56893/users/8ce2eee120f846aa8c552cde8469d994/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:14.981500 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf52ff-8717-40a9-967d-cab8f839c460 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f7874ef29f7408388914096a7b56893, user_id=8ce2eee120f846aa8c552cde8469d994, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:14.984189 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf52ff-8717-40a9-967d-cab8f839c460 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:15.002904 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1214/2422] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:44:14 2022] PUT /identity/v3/projects/2f7874ef29f7408388914096a7b56893/users/8ce2eee120f846aa8c552cde8469d994/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:44:15.007004 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54459e-5fa8-4b4f-91ed-f2ef8b436e89 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:15.017550 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54459e-5fa8-4b4f-91ed-f2ef8b436e89 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:15.082195 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80024ddf-cbe2-40ad-a7a3-be0ea509c3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:15.083084 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80024ddf-cbe2-40ad-a7a3-be0ea509c3e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:15.083294 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80024ddf-cbe2-40ad-a7a3-be0ea509c3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:15.083578 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80024ddf-cbe2-40ad-a7a3-be0ea509c3e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:15.084382 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80024ddf-cbe2-40ad-a7a3-be0ea509c3e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:15.086045 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80024ddf-cbe2-40ad-a7a3-be0ea509c3e8 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:15.098048 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1209/2423] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:44:15 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:44:15.102349 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf52ff-8717-40a9-967d-cab8f839c460 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:15.115931 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf52ff-8717-40a9-967d-cab8f839c460 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:15.121289 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4ce19-1a50-4472-812c-031ccc08a77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:15.122453 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4ce19-1a50-4472-812c-031ccc08a77e admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:15.122938 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4ce19-1a50-4472-812c-031ccc08a77e admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:15.123221 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4ce19-1a50-4472-812c-031ccc08a77e admin admin] PATH_INFO: `/v3/projects/2f7874ef29f7408388914096a7b56893/users/8ce2eee120f846aa8c552cde8469d994/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:15.126186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f4ce19-1a50-4472-812c-031ccc08a77e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f7874ef29f7408388914096a7b56893, user_id=8ce2eee120f846aa8c552cde8469d994, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:15.128978 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f4ce19-1a50-4472-812c-031ccc08a77e admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:15.142436 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1215/2424] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:44:15 2022] PUT /identity/v3/projects/2f7874ef29f7408388914096a7b56893/users/8ce2eee120f846aa8c552cde8469d994/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:44:20.698773 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e36bb6-08b8-4db2-a992-1702031246e9 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:20.699863 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e36bb6-08b8-4db2-a992-1702031246e9 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:20.700384 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e36bb6-08b8-4db2-a992-1702031246e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:20.739666 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-59e36bb6-08b8-4db2-a992-1702031246e9 None None] MFA Rules not processed for user `8ce2eee120f846aa8c552cde8469d994`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:44:20.741068 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-59e36bb6-08b8-4db2-a992-1702031246e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:20.845704 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1210/2425] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:44:20 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:44:20.861864 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4ce19-1a50-4472-812c-031ccc08a77e admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:20.864404 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-d1f4ce19-1a50-4472-812c-031ccc08a77e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:20.964074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4ce19-1a50-4472-812c-031ccc08a77e admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:20.972106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ad67c-0fbc-4a21-8bd8-dff9603cded6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:20.972861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ad67c-0fbc-4a21-8bd8-dff9603cded6 service nova] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:20.973219 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ad67c-0fbc-4a21-8bd8-dff9603cded6 service nova] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:20.973873 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ad67c-0fbc-4a21-8bd8-dff9603cded6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:21.035670 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:44:21.046297 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06ad67c-0fbc-4a21-8bd8-dff9603cded6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:21.048928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06ad67c-0fbc-4a21-8bd8-dff9603cded6 service nova] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:21.063014 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1216/2426] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:44:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:44:22.470578 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8105f6f-f90d-48e4-945d-99197af566aa None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:22.471248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8105f6f-f90d-48e4-945d-99197af566aa None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:22.471352 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8105f6f-f90d-48e4-945d-99197af566aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:22.491433 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c8105f6f-f90d-48e4-945d-99197af566aa None None] MFA Rules not processed for user `4ca9a3e7826b4928863bd83a9496205a`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:44:22.493102 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c8105f6f-f90d-48e4-945d-99197af566aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:22.513389 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1211/2427] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:44:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:44:28.714962 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ad67c-0fbc-4a21-8bd8-dff9603cded6 service nova] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:28.784320 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ad67c-0fbc-4a21-8bd8-dff9603cded6 service nova] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:28.788414 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f24af0a-ca28-4706-9d10-43a435f1b40e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:28.789099 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24af0a-ca28-4706-9d10-43a435f1b40e service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:28.789388 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24af0a-ca28-4706-9d10-43a435f1b40e service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:28.789671 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24af0a-ca28-4706-9d10-43a435f1b40e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:28.791531 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-2f24af0a-ca28-4706-9d10-43a435f1b40e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:28.856316 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f24af0a-ca28-4706-9d10-43a435f1b40e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:28.857766 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f24af0a-ca28-4706-9d10-43a435f1b40e service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:28.872417 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1217/2428] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:44:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:44:32.486334 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=74,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:44:32.667494 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8105f6f-f90d-48e4-945d-99197af566aa None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:32.670284 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c8105f6f-f90d-48e4-945d-99197af566aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:32.829584 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f24af0a-ca28-4706-9d10-43a435f1b40e service neutron] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:32.831936 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-2f24af0a-ca28-4706-9d10-43a435f1b40e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:32.867581 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f24af0a-ca28-4706-9d10-43a435f1b40e service neutron] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:32.875933 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8105f6f-f90d-48e4-945d-99197af566aa None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:32.881080 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6efa4f6-7d17-4c65-b485-ef1969cc86ce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb0ec2ef6c4748e181f1963e7dfe30b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:32.881769 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6efa4f6-7d17-4c65-b485-ef1969cc86ce service placement] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:32.882053 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6efa4f6-7d17-4c65-b485-ef1969cc86ce service placement] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:32.882511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6efa4f6-7d17-4c65-b485-ef1969cc86ce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:32.884259 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a6efa4f6-7d17-4c65-b485-ef1969cc86ce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:32.899067 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6efa4f6-7d17-4c65-b485-ef1969cc86ce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:32.900784 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6efa4f6-7d17-4c65-b485-ef1969cc86ce service placement] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:32.903526 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec50da1-b80c-4b01-b87c-4a9c5af4560a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:32.904711 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec50da1-b80c-4b01-b87c-4a9c5af4560a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:32.905005 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec50da1-b80c-4b01-b87c-4a9c5af4560a admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:32.905525 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec50da1-b80c-4b01-b87c-4a9c5af4560a admin admin] PATH_INFO: `/v3/users/c815423179f44b219e4f8572b10355c0` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:32.916233 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec50da1-b80c-4b01-b87c-4a9c5af4560a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c815423179f44b219e4f8572b10355c0)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:32.919439 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec50da1-b80c-4b01-b87c-4a9c5af4560a admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:32.934780 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1212/2429] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:44:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:44:33.048716 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-5ec50da1-b80c-4b01-b87c-4a9c5af4560a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c815423179f44b219e4f8572b10355c0'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:33.049682 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-5ec50da1-b80c-4b01-b87c-4a9c5af4560a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c815423179f44b219e4f8572b10355c0'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:33.064185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-5ec50da1-b80c-4b01-b87c-4a9c5af4560a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c815423179f44b219e4f8572b10355c0'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:33.064844 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-5ec50da1-b80c-4b01-b87c-4a9c5af4560a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c815423179f44b219e4f8572b10355c0'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:33.082020 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-5ec50da1-b80c-4b01-b87c-4a9c5af4560a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c815423179f44b219e4f8572b10355c0'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:33.126340 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1218/2430] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:44:32 2022] DELETE /identity/v3/users/c815423179f44b219e4f8572b10355c0 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:44:33.373308 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6efa4f6-7d17-4c65-b485-ef1969cc86ce service placement] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:33.387396 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6efa4f6-7d17-4c65-b485-ef1969cc86ce service placement] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:33.454344 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac5895-fff9-4f72-9ffd-04a9a8e2697a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:33.455242 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac5895-fff9-4f72-9ffd-04a9a8e2697a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:33.455524 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac5895-fff9-4f72-9ffd-04a9a8e2697a admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:33.455802 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac5895-fff9-4f72-9ffd-04a9a8e2697a admin admin] PATH_INFO: `/v3/projects/bd64a4f6222542f19ba5729f7abe12b2` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:33.457077 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac5895-fff9-4f72-9ffd-04a9a8e2697a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd64a4f6222542f19ba5729f7abe12b2)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:33.459053 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac5895-fff9-4f72-9ffd-04a9a8e2697a admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:33.513246 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4eac5895-fff9-4f72-9ffd-04a9a8e2697a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd64a4f6222542f19ba5729f7abe12b2', 'request_id': 'req-4eac5895-fff9-4f72-9ffd-04a9a8e2697a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:33.523767 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4eac5895-fff9-4f72-9ffd-04a9a8e2697a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd64a4f6222542f19ba5729f7abe12b2', 'request_id': 'req-4eac5895-fff9-4f72-9ffd-04a9a8e2697a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:33.536686 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4eac5895-fff9-4f72-9ffd-04a9a8e2697a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd64a4f6222542f19ba5729f7abe12b2', 'request_id': 'req-4eac5895-fff9-4f72-9ffd-04a9a8e2697a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:33.565968 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4eac5895-fff9-4f72-9ffd-04a9a8e2697a admin admin] The token cache is being invalidate because project bd64a4f6222542f19ba5729f7abe12b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:44:33.566490 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4eac5895-fff9-4f72-9ffd-04a9a8e2697a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:33.567477 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4eac5895-fff9-4f72-9ffd-04a9a8e2697a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:33.569679 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1213/2431] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:44:33 2022] DELETE /identity/v3/projects/bd64a4f6222542f19ba5729f7abe12b2 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:44:33.574955 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5533e9-ef6b-4d77-952b-b42a943f4a65 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:33.575255 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5533e9-ef6b-4d77-952b-b42a943f4a65 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:33.575543 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5533e9-ef6b-4d77-952b-b42a943f4a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:33.596919 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-8a5533e9-ef6b-4d77-952b-b42a943f4a65 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:44:33.598203 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-8a5533e9-ef6b-4d77-952b-b42a943f4a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:33.605618 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:44:33.695833 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1219/2432] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:44:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:44:33.704181 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64952bdf-5969-40d7-b41e-2cbc11e74dfb None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:33.704476 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64952bdf-5969-40d7-b41e-2cbc11e74dfb None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:33.704762 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64952bdf-5969-40d7-b41e-2cbc11e74dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:33.726238 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-64952bdf-5969-40d7-b41e-2cbc11e74dfb None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:44:33.727584 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-64952bdf-5969-40d7-b41e-2cbc11e74dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:33.739678 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1214/2433] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:44:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:44:33.744579 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5533e9-ef6b-4d77-952b-b42a943f4a65 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:33.756349 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5533e9-ef6b-4d77-952b-b42a943f4a65 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:33.762561 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3525a-881e-4eb3-ba47-9efdb480b628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:33.763987 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3525a-881e-4eb3-ba47-9efdb480b628 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:33.764289 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3525a-881e-4eb3-ba47-9efdb480b628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:33.764646 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3525a-881e-4eb3-ba47-9efdb480b628 admin admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:33.765531 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac3525a-881e-4eb3-ba47-9efdb480b628 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:44:33.765976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac3525a-881e-4eb3-ba47-9efdb480b628 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:33.767860 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac3525a-881e-4eb3-ba47-9efdb480b628 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:33.782124 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1220/2434] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:44:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:44:33.785467 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64952bdf-5969-40d7-b41e-2cbc11e74dfb None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:33.795834 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64952bdf-5969-40d7-b41e-2cbc11e74dfb None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:33.801368 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf29093-ecdd-4448-aab0-d7a48cf197cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:33.802109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf29093-ecdd-4448-aab0-d7a48cf197cb admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:33.802413 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf29093-ecdd-4448-aab0-d7a48cf197cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:33.802692 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf29093-ecdd-4448-aab0-d7a48cf197cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:33.803424 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf29093-ecdd-4448-aab0-d7a48cf197cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:33.805420 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf29093-ecdd-4448-aab0-d7a48cf197cb admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:33.819616 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:44:33.835772 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1215/2435] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:44:33 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:44:33.840218 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3525a-881e-4eb3-ba47-9efdb480b628 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:33.851210 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3525a-881e-4eb3-ba47-9efdb480b628 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:33.916997 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5b64c4-9142-4c39-8824-c1029ba23929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:33.917687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b64c4-9142-4c39-8824-c1029ba23929 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:33.917982 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b64c4-9142-4c39-8824-c1029ba23929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:33.918302 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b64c4-9142-4c39-8824-c1029ba23929 admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:33.919027 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5b64c4-9142-4c39-8824-c1029ba23929 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:33.920996 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5b64c4-9142-4c39-8824-c1029ba23929 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:33.943651 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1221/2436] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:44:33 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:44:33.948464 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf29093-ecdd-4448-aab0-d7a48cf197cb admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:33.959996 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf29093-ecdd-4448-aab0-d7a48cf197cb admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:33.965836 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9ff0eb-27f1-49b6-a613-a8d7d72bb0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:33.966574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ff0eb-27f1-49b6-a613-a8d7d72bb0b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:33.966921 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ff0eb-27f1-49b6-a613-a8d7d72bb0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:33.967203 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ff0eb-27f1-49b6-a613-a8d7d72bb0b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:33.968068 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9ff0eb-27f1-49b6-a613-a8d7d72bb0b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:33.969827 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9ff0eb-27f1-49b6-a613-a8d7d72bb0b8 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:33.984615 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1216/2437] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:44:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:44:33.992480 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5b64c4-9142-4c39-8824-c1029ba23929 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:34.006357 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5b64c4-9142-4c39-8824-c1029ba23929 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:34.011753 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b800d-f1b9-404c-aab4-b8370b92d300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:34.012691 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b800d-f1b9-404c-aab4-b8370b92d300 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:34.014227 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b800d-f1b9-404c-aab4-b8370b92d300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:34.014227 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b800d-f1b9-404c-aab4-b8370b92d300 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:34.014227 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b800d-f1b9-404c-aab4-b8370b92d300 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:34.017088 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b800d-f1b9-404c-aab4-b8370b92d300 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:34.029590 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1222/2438] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:44:33 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:44:34.034215 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9ff0eb-27f1-49b6-a613-a8d7d72bb0b8 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:34.049650 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9ff0eb-27f1-49b6-a613-a8d7d72bb0b8 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:34.058256 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcaa90b-722b-4938-9f05-97d1e6df29d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:34.059522 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcaa90b-722b-4938-9f05-97d1e6df29d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:34.060033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcaa90b-722b-4938-9f05-97d1e6df29d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:34.060536 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcaa90b-722b-4938-9f05-97d1e6df29d6 admin admin] PATH_INFO: `/v3/projects/6eaa58de7a6144d08f0eab80c9ad8790/users/f4886b5a1a8947af8a884d166f231e39/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:34.087379 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcaa90b-722b-4938-9f05-97d1e6df29d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eaa58de7a6144d08f0eab80c9ad8790, user_id=f4886b5a1a8947af8a884d166f231e39, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:34.090416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcaa90b-722b-4938-9f05-97d1e6df29d6 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:34.110736 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1217/2439] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:44:34 2022] PUT /identity/v3/projects/6eaa58de7a6144d08f0eab80c9ad8790/users/f4886b5a1a8947af8a884d166f231e39/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:44:34.119169 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c30286-3438-4544-9a9f-2eabfb03e622 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:34.119514 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c30286-3438-4544-9a9f-2eabfb03e622 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:34.119730 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c30286-3438-4544-9a9f-2eabfb03e622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:34.160887 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-91c30286-3438-4544-9a9f-2eabfb03e622 None None] MFA Rules not processed for user `f4886b5a1a8947af8a884d166f231e39`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:44:34.162583 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-91c30286-3438-4544-9a9f-2eabfb03e622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:34.243517 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1223/2440] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:44:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:44:34.254682 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcaa90b-722b-4938-9f05-97d1e6df29d6 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:34.256542 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0dcaa90b-722b-4938-9f05-97d1e6df29d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:34.375847 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcaa90b-722b-4938-9f05-97d1e6df29d6 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:34.382582 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3dc6b-4bcb-4913-8f51-55e633c6e7ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:34.383265 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3dc6b-4bcb-4913-8f51-55e633c6e7ba service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:34.383555 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3dc6b-4bcb-4913-8f51-55e633c6e7ba service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:34.383833 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3dc6b-4bcb-4913-8f51-55e633c6e7ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:34.395499 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3dc6b-4bcb-4913-8f51-55e633c6e7ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:34.395699 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3dc6b-4bcb-4913-8f51-55e633c6e7ba service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:34.408077 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1218/2441] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:44:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:44:41.927231 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c30286-3438-4544-9a9f-2eabfb03e622 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:41.929595 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-91c30286-3438-4544-9a9f-2eabfb03e622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:41.945425 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:44:42.004215 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c30286-3438-4544-9a9f-2eabfb03e622 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:42.009554 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa9d7e5-d35e-469d-b7e4-dc7a5a4b77a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:42.010278 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa9d7e5-d35e-469d-b7e4-dc7a5a4b77a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:42.010562 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa9d7e5-d35e-469d-b7e4-dc7a5a4b77a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:42.010845 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa9d7e5-d35e-469d-b7e4-dc7a5a4b77a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:42.012738 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-2fa9d7e5-d35e-469d-b7e4-dc7a5a4b77a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:42.085285 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa9d7e5-d35e-469d-b7e4-dc7a5a4b77a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:42.086782 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa9d7e5-d35e-469d-b7e4-dc7a5a4b77a6 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:42.098412 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1224/2442] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:44:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:44:43.005423 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3dc6b-4bcb-4913-8f51-55e633c6e7ba service nova] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:43.028545 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3dc6b-4bcb-4913-8f51-55e633c6e7ba service nova] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:43.033847 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf7d50-afd8-4641-a97d-813421771d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:43.035238 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf7d50-afd8-4641-a97d-813421771d3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:43.035671 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf7d50-afd8-4641-a97d-813421771d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:43.036109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf7d50-afd8-4641-a97d-813421771d3e admin admin] PATH_INFO: `/v3/users/f4886b5a1a8947af8a884d166f231e39` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:43.038111 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf7d50-afd8-4641-a97d-813421771d3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4886b5a1a8947af8a884d166f231e39)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:43.041531 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf7d50-afd8-4641-a97d-813421771d3e admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:43.118454 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4ddf7d50-afd8-4641-a97d-813421771d3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4886b5a1a8947af8a884d166f231e39'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:43.120627 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4ddf7d50-afd8-4641-a97d-813421771d3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4886b5a1a8947af8a884d166f231e39'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:43.122754 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4ddf7d50-afd8-4641-a97d-813421771d3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4886b5a1a8947af8a884d166f231e39'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:43.148507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4ddf7d50-afd8-4641-a97d-813421771d3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4886b5a1a8947af8a884d166f231e39'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:43.160961 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4ddf7d50-afd8-4641-a97d-813421771d3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4886b5a1a8947af8a884d166f231e39'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:43.189354 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4ee1f-b374-42ab-8db7-e1e243c6974c None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:43.189891 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4ee1f-b374-42ab-8db7-e1e243c6974c None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:43.190129 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4ee1f-b374-42ab-8db7-e1e243c6974c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:43.190322 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1219/2443] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:44:43 2022] DELETE /identity/v3/users/f4886b5a1a8947af8a884d166f231e39 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:44:43.212443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-eda4ee1f-b374-42ab-8db7-e1e243c6974c None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:44:43.213665 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-eda4ee1f-b374-42ab-8db7-e1e243c6974c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:43.282564 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1225/2444] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:44:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:44:43.293756 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf7d50-afd8-4641-a97d-813421771d3e admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:43.295430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-4ddf7d50-afd8-4641-a97d-813421771d3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:43.315779 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:44:43.374773 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf7d50-afd8-4641-a97d-813421771d3e admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:43.378594 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0f02c-364e-4afa-a8e2-7adc74cf3024 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:43.379259 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0f02c-364e-4afa-a8e2-7adc74cf3024 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:43.379546 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0f02c-364e-4afa-a8e2-7adc74cf3024 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:43.379830 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0f02c-364e-4afa-a8e2-7adc74cf3024 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:43.392207 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc0f02c-364e-4afa-a8e2-7adc74cf3024 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:43.393803 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc0f02c-364e-4afa-a8e2-7adc74cf3024 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:43.405589 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1220/2445] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:44:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:44:43.815714 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda4ee1f-b374-42ab-8db7-e1e243c6974c None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:43.817652 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-eda4ee1f-b374-42ab-8db7-e1e243c6974c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:43.854423 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda4ee1f-b374-42ab-8db7-e1e243c6974c None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:43.875584 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ffe312-bad2-4406-a7f9-2a065940078c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:43.878061 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffe312-bad2-4406-a7f9-2a065940078c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:43.878373 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffe312-bad2-4406-a7f9-2a065940078c admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:43.882363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffe312-bad2-4406-a7f9-2a065940078c admin admin] PATH_INFO: `/v3/projects/6eaa58de7a6144d08f0eab80c9ad8790` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:43.890731 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ffe312-bad2-4406-a7f9-2a065940078c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6eaa58de7a6144d08f0eab80c9ad8790)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:43.893167 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ffe312-bad2-4406-a7f9-2a065940078c admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:43.991575 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-15ffe312-bad2-4406-a7f9-2a065940078c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6eaa58de7a6144d08f0eab80c9ad8790', 'request_id': 'req-15ffe312-bad2-4406-a7f9-2a065940078c'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:43.996223 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:44:44.006355 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-15ffe312-bad2-4406-a7f9-2a065940078c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6eaa58de7a6144d08f0eab80c9ad8790', 'request_id': 'req-15ffe312-bad2-4406-a7f9-2a065940078c'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:44.018816 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-15ffe312-bad2-4406-a7f9-2a065940078c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6eaa58de7a6144d08f0eab80c9ad8790', 'request_id': 'req-15ffe312-bad2-4406-a7f9-2a065940078c'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:44.040876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-15ffe312-bad2-4406-a7f9-2a065940078c admin admin] The token cache is being invalidate because project 6eaa58de7a6144d08f0eab80c9ad8790 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:44:44.041179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-15ffe312-bad2-4406-a7f9-2a065940078c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:44.042243 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-15ffe312-bad2-4406-a7f9-2a065940078c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:44.046578 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1226/2446] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:44:43 2022] DELETE /identity/v3/projects/6eaa58de7a6144d08f0eab80c9ad8790 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:44:44.052821 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0bb22-634f-4f00-9add-f36f2e28b8cb None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:44.053730 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0bb22-634f-4f00-9add-f36f2e28b8cb None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:44.055117 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0bb22-634f-4f00-9add-f36f2e28b8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:44.079387 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-f6a0bb22-634f-4f00-9add-f36f2e28b8cb None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:44:44.081002 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f6a0bb22-634f-4f00-9add-f36f2e28b8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:44.185245 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1221/2447] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:44:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:44:44.196755 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656c8c0-d9d9-4339-b7dd-d1ec2b106c08 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:44.197198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656c8c0-d9d9-4339-b7dd-d1ec2b106c08 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:44.197318 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656c8c0-d9d9-4339-b7dd-d1ec2b106c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:44.226279 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-b656c8c0-d9d9-4339-b7dd-d1ec2b106c08 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:44:44.227513 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-b656c8c0-d9d9-4339-b7dd-d1ec2b106c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:44.241282 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1227/2448] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:44:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:44:44.247320 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0bb22-634f-4f00-9add-f36f2e28b8cb None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:44.259859 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0bb22-634f-4f00-9add-f36f2e28b8cb None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:44.265963 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b39ad4-4c28-462a-a955-b4c76b57aa70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:44.266929 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b39ad4-4c28-462a-a955-b4c76b57aa70 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:44.267432 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b39ad4-4c28-462a-a955-b4c76b57aa70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:44.267947 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b39ad4-4c28-462a-a955-b4c76b57aa70 admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:44.268946 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b39ad4-4c28-462a-a955-b4c76b57aa70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:44:44.269449 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b39ad4-4c28-462a-a955-b4c76b57aa70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:44.271774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b39ad4-4c28-462a-a955-b4c76b57aa70 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:44.285866 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1222/2449] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:44:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:44:44.290044 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656c8c0-d9d9-4339-b7dd-d1ec2b106c08 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:44.301658 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656c8c0-d9d9-4339-b7dd-d1ec2b106c08 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:44.307681 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2682373-0e48-4d98-ac0d-6d789b05e443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:44.309003 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2682373-0e48-4d98-ac0d-6d789b05e443 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:44.309716 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2682373-0e48-4d98-ac0d-6d789b05e443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:44.310139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2682373-0e48-4d98-ac0d-6d789b05e443 admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:44.312057 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2682373-0e48-4d98-ac0d-6d789b05e443 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:44.315452 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2682373-0e48-4d98-ac0d-6d789b05e443 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:44.371977 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1228/2450] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:44:44 2022] POST /identity/v3/projects => generated 367 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:44:44.377536 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b39ad4-4c28-462a-a955-b4c76b57aa70 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:44.403190 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b39ad4-4c28-462a-a955-b4c76b57aa70 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:44.515003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e91d02-bc85-4364-973e-ddab1296c817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:44.515895 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e91d02-bc85-4364-973e-ddab1296c817 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:44.516182 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e91d02-bc85-4364-973e-ddab1296c817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:44.516460 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e91d02-bc85-4364-973e-ddab1296c817 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:44.517168 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e91d02-bc85-4364-973e-ddab1296c817 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:44.519485 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e91d02-bc85-4364-973e-ddab1296c817 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:44.556001 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1223/2451] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:44:44 2022] POST /identity/v3/users => generated 284 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:44:44.561807 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2682373-0e48-4d98-ac0d-6d789b05e443 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:44.579246 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2682373-0e48-4d98-ac0d-6d789b05e443 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:44.586559 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f04df-380b-41bc-ab1f-ccb657c0b5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:44.587255 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f04df-380b-41bc-ab1f-ccb657c0b5cf admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:44.587702 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f04df-380b-41bc-ab1f-ccb657c0b5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:44.588124 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f04df-380b-41bc-ab1f-ccb657c0b5cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:44.588935 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473f04df-380b-41bc-ab1f-ccb657c0b5cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:44.590758 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473f04df-380b-41bc-ab1f-ccb657c0b5cf admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:44.612036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1229/2452] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:44:44.613607 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e91d02-bc85-4364-973e-ddab1296c817 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:44.625248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e91d02-bc85-4364-973e-ddab1296c817 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:44.631564 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76608d40-e01d-43d6-83f6-a2de34b308f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:44.632419 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76608d40-e01d-43d6-83f6-a2de34b308f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:44.632750 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76608d40-e01d-43d6-83f6-a2de34b308f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:44.633088 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76608d40-e01d-43d6-83f6-a2de34b308f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:44.634105 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76608d40-e01d-43d6-83f6-a2de34b308f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:44.636041 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76608d40-e01d-43d6-83f6-a2de34b308f2 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:44.660409 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1224/2453] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:44:44.661679 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f04df-380b-41bc-ab1f-ccb657c0b5cf admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:44.673063 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f04df-380b-41bc-ab1f-ccb657c0b5cf admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:44.682256 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540475a-dd70-4599-b57b-d46da57d2d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:44.683522 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540475a-dd70-4599-b57b-d46da57d2d9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:44.683962 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540475a-dd70-4599-b57b-d46da57d2d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:44.684387 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540475a-dd70-4599-b57b-d46da57d2d9c admin admin] PATH_INFO: `/v3/projects/1afcb3ba6f8a498d8061ad7ab16e8edc/users/06e062f66ada4639b7a8edeaf6f95bf1/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:44.704941 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4540475a-dd70-4599-b57b-d46da57d2d9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1afcb3ba6f8a498d8061ad7ab16e8edc, user_id=06e062f66ada4639b7a8edeaf6f95bf1, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:44.708779 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4540475a-dd70-4599-b57b-d46da57d2d9c admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:44.741877 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1230/2454] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:44:44 2022] PUT /identity/v3/projects/1afcb3ba6f8a498d8061ad7ab16e8edc/users/06e062f66ada4639b7a8edeaf6f95bf1/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:44:44.744594 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62e930-cc72-492f-8b46-46336933f106 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:44.745593 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62e930-cc72-492f-8b46-46336933f106 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:44.746324 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62e930-cc72-492f-8b46-46336933f106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:44.853463 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-8a62e930-cc72-492f-8b46-46336933f106 None None] MFA Rules not processed for user `06e062f66ada4639b7a8edeaf6f95bf1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:44:44.864138 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8a62e930-cc72-492f-8b46-46336933f106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:44.882811 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:44:44.976028 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1225/2455] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:44:44 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:44:44.989500 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540475a-dd70-4599-b57b-d46da57d2d9c admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:44.993160 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-4540475a-dd70-4599-b57b-d46da57d2d9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:45.076641 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540475a-dd70-4599-b57b-d46da57d2d9c admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:45.081698 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45453a79-a7ba-4d65-b610-0c861db6183a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:45.082674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45453a79-a7ba-4d65-b610-0c861db6183a service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:45.083923 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45453a79-a7ba-4d65-b610-0c861db6183a service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:45.084967 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45453a79-a7ba-4d65-b610-0c861db6183a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:45.099357 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45453a79-a7ba-4d65-b610-0c861db6183a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:45.101485 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45453a79-a7ba-4d65-b610-0c861db6183a service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:45.127069 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1231/2456] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:44:44 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:44:46.680624 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62e930-cc72-492f-8b46-46336933f106 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:46.682818 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8a62e930-cc72-492f-8b46-46336933f106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:46.776137 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62e930-cc72-492f-8b46-46336933f106 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:46.785767 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe7053-c890-49ce-b3e5-58d2ab45ad4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:46.787597 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe7053-c890-49ce-b3e5-58d2ab45ad4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:46.788293 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe7053-c890-49ce-b3e5-58d2ab45ad4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:46.788813 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe7053-c890-49ce-b3e5-58d2ab45ad4f admin admin] PATH_INFO: `/v3/users/e1635576dfe242f88ec1abb7be6359fb` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:46.791359 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efe7053-c890-49ce-b3e5-58d2ab45ad4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1635576dfe242f88ec1abb7be6359fb)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:46.795192 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efe7053-c890-49ce-b3e5-58d2ab45ad4f admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:46.866449 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2efe7053-c890-49ce-b3e5-58d2ab45ad4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1635576dfe242f88ec1abb7be6359fb'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:46.867186 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2efe7053-c890-49ce-b3e5-58d2ab45ad4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1635576dfe242f88ec1abb7be6359fb'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:46.868144 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2efe7053-c890-49ce-b3e5-58d2ab45ad4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1635576dfe242f88ec1abb7be6359fb'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:46.884338 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2efe7053-c890-49ce-b3e5-58d2ab45ad4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1635576dfe242f88ec1abb7be6359fb'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:46.894148 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2efe7053-c890-49ce-b3e5-58d2ab45ad4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1635576dfe242f88ec1abb7be6359fb'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:46.920672 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1226/2457] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:44:46 2022] DELETE /identity/v3/users/e1635576dfe242f88ec1abb7be6359fb => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:44:47.153652 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45453a79-a7ba-4d65-b610-0c861db6183a service cinder] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:47.184035 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45453a79-a7ba-4d65-b610-0c861db6183a service cinder] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:47.254754 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212528a2-aa70-417c-bc2d-64b651a9e878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:47.255722 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212528a2-aa70-417c-bc2d-64b651a9e878 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:47.256081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212528a2-aa70-417c-bc2d-64b651a9e878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:47.256432 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212528a2-aa70-417c-bc2d-64b651a9e878 admin admin] PATH_INFO: `/v3/projects/e8e1414ffa3548938432aed200c75736` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:47.257723 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212528a2-aa70-417c-bc2d-64b651a9e878 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8e1414ffa3548938432aed200c75736)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:47.259698 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212528a2-aa70-417c-bc2d-64b651a9e878 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:47.287275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:44:47.312212 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-212528a2-aa70-417c-bc2d-64b651a9e878 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8e1414ffa3548938432aed200c75736', 'request_id': 'req-212528a2-aa70-417c-bc2d-64b651a9e878'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:47.320557 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-212528a2-aa70-417c-bc2d-64b651a9e878 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8e1414ffa3548938432aed200c75736', 'request_id': 'req-212528a2-aa70-417c-bc2d-64b651a9e878'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:47.329835 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-212528a2-aa70-417c-bc2d-64b651a9e878 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8e1414ffa3548938432aed200c75736', 'request_id': 'req-212528a2-aa70-417c-bc2d-64b651a9e878'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:47.345540 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-212528a2-aa70-417c-bc2d-64b651a9e878 admin admin] The token cache is being invalidate because project e8e1414ffa3548938432aed200c75736 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:44:47.346174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-212528a2-aa70-417c-bc2d-64b651a9e878 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:47.347367 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-212528a2-aa70-417c-bc2d-64b651a9e878 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:47.349456 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1232/2458] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:44:47 2022] DELETE /identity/v3/projects/e8e1414ffa3548938432aed200c75736 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:44:47.354781 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54403bea-2210-4878-9e3d-c1f07bcadb51 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:47.355073 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54403bea-2210-4878-9e3d-c1f07bcadb51 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:47.355355 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54403bea-2210-4878-9e3d-c1f07bcadb51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:47.372937 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-54403bea-2210-4878-9e3d-c1f07bcadb51 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:44:47.374683 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-54403bea-2210-4878-9e3d-c1f07bcadb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:47.401136 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:44:47.435580 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1227/2459] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:44:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:44:47.444660 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abfbd32-4489-4319-b3c0-2f8115bfc1ba None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:47.444971 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abfbd32-4489-4319-b3c0-2f8115bfc1ba None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:47.445274 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abfbd32-4489-4319-b3c0-2f8115bfc1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:47.464312 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-7abfbd32-4489-4319-b3c0-2f8115bfc1ba None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:44:47.465518 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-7abfbd32-4489-4319-b3c0-2f8115bfc1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:47.473316 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1233/2460] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:44:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:44:47.477534 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54403bea-2210-4878-9e3d-c1f07bcadb51 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:47.487825 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54403bea-2210-4878-9e3d-c1f07bcadb51 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:47.492441 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a6349-80a7-4891-bcb0-217ca840a78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:47.493355 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a6349-80a7-4891-bcb0-217ca840a78e admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:47.493640 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a6349-80a7-4891-bcb0-217ca840a78e admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:47.493920 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a6349-80a7-4891-bcb0-217ca840a78e admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:47.494711 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882a6349-80a7-4891-bcb0-217ca840a78e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:44:47.495021 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882a6349-80a7-4891-bcb0-217ca840a78e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:47.496632 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882a6349-80a7-4891-bcb0-217ca840a78e admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:47.508915 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1228/2461] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:44:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:44:47.513477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abfbd32-4489-4319-b3c0-2f8115bfc1ba None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:47.523889 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abfbd32-4489-4319-b3c0-2f8115bfc1ba None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:47.528858 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22174bc1-3b6f-44ec-8c0e-5a3fa6f2d12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:47.529511 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22174bc1-3b6f-44ec-8c0e-5a3fa6f2d12b admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:47.529807 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22174bc1-3b6f-44ec-8c0e-5a3fa6f2d12b admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:47.530200 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22174bc1-3b6f-44ec-8c0e-5a3fa6f2d12b admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:47.530899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22174bc1-3b6f-44ec-8c0e-5a3fa6f2d12b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:47.532793 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22174bc1-3b6f-44ec-8c0e-5a3fa6f2d12b admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:47.558406 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1234/2462] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:44:47 2022] POST /identity/v3/projects => generated 399 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:44:47.563074 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a6349-80a7-4891-bcb0-217ca840a78e admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:47.573482 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a6349-80a7-4891-bcb0-217ca840a78e admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:47.643120 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1901e-1a25-4c64-a3ff-47352156ea03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:47.643810 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1901e-1a25-4c64-a3ff-47352156ea03 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:47.644093 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1901e-1a25-4c64-a3ff-47352156ea03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:47.644515 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1901e-1a25-4c64-a3ff-47352156ea03 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:47.646072 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf1901e-1a25-4c64-a3ff-47352156ea03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:47.647417 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf1901e-1a25-4c64-a3ff-47352156ea03 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:47.674111 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1229/2463] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:44:47 2022] POST /identity/v3/users => generated 300 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:44:47.678067 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22174bc1-3b6f-44ec-8c0e-5a3fa6f2d12b admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:47.693382 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22174bc1-3b6f-44ec-8c0e-5a3fa6f2d12b admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:47.699415 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e34a2b-7b8c-4540-b294-b5eaff5ca2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:47.700248 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e34a2b-7b8c-4540-b294-b5eaff5ca2ce admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:47.700743 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e34a2b-7b8c-4540-b294-b5eaff5ca2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:47.701135 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e34a2b-7b8c-4540-b294-b5eaff5ca2ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:47.702308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e34a2b-7b8c-4540-b294-b5eaff5ca2ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:47.704287 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e34a2b-7b8c-4540-b294-b5eaff5ca2ce admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:47.721325 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1235/2464] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:44:47.723327 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1901e-1a25-4c64-a3ff-47352156ea03 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:47.734272 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1901e-1a25-4c64-a3ff-47352156ea03 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:47.739147 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be0423-9abe-41a7-879d-684c784d9bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:47.739840 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be0423-9abe-41a7-879d-684c784d9bef admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:47.740123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be0423-9abe-41a7-879d-684c784d9bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:47.740406 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be0423-9abe-41a7-879d-684c784d9bef admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:47.741156 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72be0423-9abe-41a7-879d-684c784d9bef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:47.743115 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72be0423-9abe-41a7-879d-684c784d9bef admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:47.757641 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1230/2465] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:44:47.761908 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e34a2b-7b8c-4540-b294-b5eaff5ca2ce admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:47.773425 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e34a2b-7b8c-4540-b294-b5eaff5ca2ce admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:47.782880 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224bdfb-7af9-47ff-846e-18bcdbdd7411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:47.783788 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224bdfb-7af9-47ff-846e-18bcdbdd7411 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:47.784079 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224bdfb-7af9-47ff-846e-18bcdbdd7411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:47.784352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224bdfb-7af9-47ff-846e-18bcdbdd7411 admin admin] PATH_INFO: `/v3/projects/23d191c6d2a34166b6061bbc3e5b1499/users/9f5198c6bfda40e6a63d690939a37c8c/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:47.801564 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224bdfb-7af9-47ff-846e-18bcdbdd7411 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23d191c6d2a34166b6061bbc3e5b1499, user_id=9f5198c6bfda40e6a63d690939a37c8c, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:47.804284 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224bdfb-7af9-47ff-846e-18bcdbdd7411 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:47.818730 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1236/2466] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:44:47 2022] PUT /identity/v3/projects/23d191c6d2a34166b6061bbc3e5b1499/users/9f5198c6bfda40e6a63d690939a37c8c/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:44:47.830099 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f0cf8-7483-40dc-ba20-90ceca324c35 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:47.830714 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f0cf8-7483-40dc-ba20-90ceca324c35 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:47.831195 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f0cf8-7483-40dc-ba20-90ceca324c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:47.866321 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-4a7f0cf8-7483-40dc-ba20-90ceca324c35 None None] MFA Rules not processed for user `9f5198c6bfda40e6a63d690939a37c8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:44:47.867562 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-4a7f0cf8-7483-40dc-ba20-90ceca324c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:47.951975 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1231/2467] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:44:47 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:44:47.956866 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224bdfb-7af9-47ff-846e-18bcdbdd7411 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:48.021167 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224bdfb-7af9-47ff-846e-18bcdbdd7411 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:48.026871 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e14a6-052a-48f6-a645-61875b8831d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:48.027543 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e14a6-052a-48f6-a645-61875b8831d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:48.027828 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e14a6-052a-48f6-a645-61875b8831d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:48.028105 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e14a6-052a-48f6-a645-61875b8831d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:48.028818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5e14a6-052a-48f6-a645-61875b8831d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:48.031337 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5e14a6-052a-48f6-a645-61875b8831d5 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:48.072393 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1237/2468] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:44:47 2022] POST /identity/v3/projects => generated 399 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:44:48.076695 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f0cf8-7483-40dc-ba20-90ceca324c35 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:48.088255 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f0cf8-7483-40dc-ba20-90ceca324c35 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:48.114833 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:44:48.157332 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb6fd1d-1167-4e6b-aec5-d011c8938b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:48.157841 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb6fd1d-1167-4e6b-aec5-d011c8938b07 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:48.158224 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb6fd1d-1167-4e6b-aec5-d011c8938b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:48.158610 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb6fd1d-1167-4e6b-aec5-d011c8938b07 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:48.159444 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb6fd1d-1167-4e6b-aec5-d011c8938b07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:48.161700 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb6fd1d-1167-4e6b-aec5-d011c8938b07 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:48.189925 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1232/2469] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:44:48 2022] POST /identity/v3/users => generated 306 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:44:48.190430 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e14a6-052a-48f6-a645-61875b8831d5 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:48.201476 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e14a6-052a-48f6-a645-61875b8831d5 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:48.206985 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc0df0e-4850-40c2-91b4-8b6ae72f236a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:48.207682 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0df0e-4850-40c2-91b4-8b6ae72f236a admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:48.207947 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0df0e-4850-40c2-91b4-8b6ae72f236a admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:48.208223 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0df0e-4850-40c2-91b4-8b6ae72f236a admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:48.208961 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc0df0e-4850-40c2-91b4-8b6ae72f236a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:48.210706 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc0df0e-4850-40c2-91b4-8b6ae72f236a admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:48.224623 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1238/2470] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:44:48.228961 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb6fd1d-1167-4e6b-aec5-d011c8938b07 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:48.246298 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb6fd1d-1167-4e6b-aec5-d011c8938b07 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:48.251840 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9849365-e910-4f7c-b430-15e976545bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:48.252887 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9849365-e910-4f7c-b430-15e976545bdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:48.253668 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9849365-e910-4f7c-b430-15e976545bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:48.253837 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9849365-e910-4f7c-b430-15e976545bdc admin admin] PATH_INFO: `/v3/projects/7c74cd1f44754d9991eec7ecdcfd3b04/users/f44d30bba9364527a0db0a7155356cd8/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:48.271172 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9849365-e910-4f7c-b430-15e976545bdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c74cd1f44754d9991eec7ecdcfd3b04, user_id=f44d30bba9364527a0db0a7155356cd8, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:48.274389 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9849365-e910-4f7c-b430-15e976545bdc admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:48.294278 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1233/2471] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:44:48 2022] PUT /identity/v3/projects/7c74cd1f44754d9991eec7ecdcfd3b04/users/f44d30bba9364527a0db0a7155356cd8/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:44:48.298426 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc0df0e-4850-40c2-91b4-8b6ae72f236a admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:48.311275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc0df0e-4850-40c2-91b4-8b6ae72f236a admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:48.331083 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:44:48.385417 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047f0e13-afc8-45dc-8e52-d9f51fc96378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:48.386135 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f0e13-afc8-45dc-8e52-d9f51fc96378 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:48.386601 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f0e13-afc8-45dc-8e52-d9f51fc96378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:48.386883 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f0e13-afc8-45dc-8e52-d9f51fc96378 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:48.387625 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047f0e13-afc8-45dc-8e52-d9f51fc96378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:48.390368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047f0e13-afc8-45dc-8e52-d9f51fc96378 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:48.407292 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1239/2472] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:44:48.407405 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9849365-e910-4f7c-b430-15e976545bdc admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:48.423505 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9849365-e910-4f7c-b430-15e976545bdc admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:48.432014 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090d32f-2a94-4e5e-9a83-11f354737520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:48.432915 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090d32f-2a94-4e5e-9a83-11f354737520 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:48.433398 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090d32f-2a94-4e5e-9a83-11f354737520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:48.433964 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090d32f-2a94-4e5e-9a83-11f354737520 admin admin] PATH_INFO: `/v3/projects/7c74cd1f44754d9991eec7ecdcfd3b04/users/f44d30bba9364527a0db0a7155356cd8/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:48.438568 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c090d32f-2a94-4e5e-9a83-11f354737520 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c74cd1f44754d9991eec7ecdcfd3b04, user_id=f44d30bba9364527a0db0a7155356cd8, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:48.445170 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c090d32f-2a94-4e5e-9a83-11f354737520 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:48.466129 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1234/2473] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:44:48 2022] PUT /identity/v3/projects/7c74cd1f44754d9991eec7ecdcfd3b04/users/f44d30bba9364527a0db0a7155356cd8/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:44:48.481886 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61104831-0efa-4a26-b9f6-3d9aab7f3fa0 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:48.482212 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61104831-0efa-4a26-b9f6-3d9aab7f3fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:48.482494 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61104831-0efa-4a26-b9f6-3d9aab7f3fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:48.540694 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-61104831-0efa-4a26-b9f6-3d9aab7f3fa0 None None] MFA Rules not processed for user `f44d30bba9364527a0db0a7155356cd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:44:48.541939 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-61104831-0efa-4a26-b9f6-3d9aab7f3fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:48.617282 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1240/2474] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:44:48 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:44:48.628655 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090d32f-2a94-4e5e-9a83-11f354737520 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:48.630464 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c090d32f-2a94-4e5e-9a83-11f354737520 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:48.690109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090d32f-2a94-4e5e-9a83-11f354737520 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:48.694403 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66096496-65f6-48ac-bc80-bf52a8187250 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:48.695382 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66096496-65f6-48ac-bc80-bf52a8187250 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:48.695760 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66096496-65f6-48ac-bc80-bf52a8187250 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:48.696103 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66096496-65f6-48ac-bc80-bf52a8187250 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:48.707922 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66096496-65f6-48ac-bc80-bf52a8187250 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:48.709603 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66096496-65f6-48ac-bc80-bf52a8187250 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:48.722996 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1235/2475] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:44:48 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:44:48.804075 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61104831-0efa-4a26-b9f6-3d9aab7f3fa0 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:48.814451 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61104831-0efa-4a26-b9f6-3d9aab7f3fa0 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:48.818359 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e3b6b-e809-4a3c-9639-46e7d8753928 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:48.819125 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e3b6b-e809-4a3c-9639-46e7d8753928 service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:48.819733 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e3b6b-e809-4a3c-9639-46e7d8753928 service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:48.820063 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e3b6b-e809-4a3c-9639-46e7d8753928 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:48.883725 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44e3b6b-e809-4a3c-9639-46e7d8753928 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:48.885287 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44e3b6b-e809-4a3c-9639-46e7d8753928 service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:48.895459 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1241/2476] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:44:48 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:44:53.076329 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66096496-65f6-48ac-bc80-bf52a8187250 service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:53.141126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66096496-65f6-48ac-bc80-bf52a8187250 service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:53.145678 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1c52c8-0d8d-4497-9881-c974975a23fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:53.146582 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c52c8-0d8d-4497-9881-c974975a23fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:53.147021 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c52c8-0d8d-4497-9881-c974975a23fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:53.147297 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c52c8-0d8d-4497-9881-c974975a23fe admin admin] PATH_INFO: `/v3/users/9f5198c6bfda40e6a63d690939a37c8c` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:53.148539 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1c52c8-0d8d-4497-9881-c974975a23fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f5198c6bfda40e6a63d690939a37c8c)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:53.150510 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1c52c8-0d8d-4497-9881-c974975a23fe admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:53.176186 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:44:53.204759 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-bf1c52c8-0d8d-4497-9881-c974975a23fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f5198c6bfda40e6a63d690939a37c8c'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:53.205463 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-bf1c52c8-0d8d-4497-9881-c974975a23fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f5198c6bfda40e6a63d690939a37c8c'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:53.206166 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-bf1c52c8-0d8d-4497-9881-c974975a23fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f5198c6bfda40e6a63d690939a37c8c'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:53.219081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-bf1c52c8-0d8d-4497-9881-c974975a23fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f5198c6bfda40e6a63d690939a37c8c'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:53.233265 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-bf1c52c8-0d8d-4497-9881-c974975a23fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f5198c6bfda40e6a63d690939a37c8c'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:53.247079 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1236/2477] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:44:53 2022] DELETE /identity/v3/users/9f5198c6bfda40e6a63d690939a37c8c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:44:53.252301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b08bbd-d7f9-46b4-99f0-5888e1423cf7 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:53.252804 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b08bbd-d7f9-46b4-99f0-5888e1423cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:53.252930 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b08bbd-d7f9-46b4-99f0-5888e1423cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:53.275420 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-b2b08bbd-d7f9-46b4-99f0-5888e1423cf7 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:44:53.276769 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-b2b08bbd-d7f9-46b4-99f0-5888e1423cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:53.310572 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:44:53.348106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1242/2478] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:44:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:44:53.357924 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1c52c8-0d8d-4497-9881-c974975a23fe admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:53.359376 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-bf1c52c8-0d8d-4497-9881-c974975a23fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:53.420060 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1c52c8-0d8d-4497-9881-c974975a23fe admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:53.423801 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f507c-2a5e-4ebd-9136-8f7de3f0c6b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:53.424464 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f507c-2a5e-4ebd-9136-8f7de3f0c6b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:53.424735 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f507c-2a5e-4ebd-9136-8f7de3f0c6b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:53.425007 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f507c-2a5e-4ebd-9136-8f7de3f0c6b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:53.434588 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19f507c-2a5e-4ebd-9136-8f7de3f0c6b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:53.436535 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19f507c-2a5e-4ebd-9136-8f7de3f0c6b1 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:53.446698 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1237/2479] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:44:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:44:53.803952 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b08bbd-d7f9-46b4-99f0-5888e1423cf7 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:53.805611 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-b2b08bbd-d7f9-46b4-99f0-5888e1423cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:53.823914 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b08bbd-d7f9-46b4-99f0-5888e1423cf7 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:53.830396 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a196c1-4bff-4f7f-b1cb-b03f8bb40516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:53.831511 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a196c1-4bff-4f7f-b1cb-b03f8bb40516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:53.831848 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a196c1-4bff-4f7f-b1cb-b03f8bb40516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:53.832220 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a196c1-4bff-4f7f-b1cb-b03f8bb40516 admin admin] PATH_INFO: `/v3/projects/23d191c6d2a34166b6061bbc3e5b1499` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:53.833615 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a196c1-4bff-4f7f-b1cb-b03f8bb40516 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23d191c6d2a34166b6061bbc3e5b1499)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:53.835907 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a196c1-4bff-4f7f-b1cb-b03f8bb40516 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:53.861132 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f507c-2a5e-4ebd-9136-8f7de3f0c6b1 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:53.862797 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c19f507c-2a5e-4ebd-9136-8f7de3f0c6b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:53.877329 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f507c-2a5e-4ebd-9136-8f7de3f0c6b1 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:53.882443 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340cc0f-9bdf-45c3-8be5-5c76d6611e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:53.883342 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340cc0f-9bdf-45c3-8be5-5c76d6611e25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:53.883626 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340cc0f-9bdf-45c3-8be5-5c76d6611e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:53.883934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340cc0f-9bdf-45c3-8be5-5c76d6611e25 admin admin] PATH_INFO: `/v3/users/06e062f66ada4639b7a8edeaf6f95bf1` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:53.885183 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1340cc0f-9bdf-45c3-8be5-5c76d6611e25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06e062f66ada4639b7a8edeaf6f95bf1)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:53.887572 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1340cc0f-9bdf-45c3-8be5-5c76d6611e25 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:53.921880 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-45a196c1-4bff-4f7f-b1cb-b03f8bb40516 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23d191c6d2a34166b6061bbc3e5b1499', 'request_id': 'req-45a196c1-4bff-4f7f-b1cb-b03f8bb40516'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:53.932965 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-45a196c1-4bff-4f7f-b1cb-b03f8bb40516 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23d191c6d2a34166b6061bbc3e5b1499', 'request_id': 'req-45a196c1-4bff-4f7f-b1cb-b03f8bb40516'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:53.944793 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-45a196c1-4bff-4f7f-b1cb-b03f8bb40516 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23d191c6d2a34166b6061bbc3e5b1499', 'request_id': 'req-45a196c1-4bff-4f7f-b1cb-b03f8bb40516'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:53.963434 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-45a196c1-4bff-4f7f-b1cb-b03f8bb40516 admin admin] The token cache is being invalidate because project 23d191c6d2a34166b6061bbc3e5b1499 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:44:53.963907 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-45a196c1-4bff-4f7f-b1cb-b03f8bb40516 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:53.964593 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-45a196c1-4bff-4f7f-b1cb-b03f8bb40516 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:53.968059 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1243/2480] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:44:53 2022] DELETE /identity/v3/projects/23d191c6d2a34166b6061bbc3e5b1499 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:44:53.972672 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a196c1-4bff-4f7f-b1cb-b03f8bb40516 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:53.975158 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-45a196c1-4bff-4f7f-b1cb-b03f8bb40516 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:53.976896 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1340cc0f-9bdf-45c3-8be5-5c76d6611e25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06e062f66ada4639b7a8edeaf6f95bf1'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:53.977915 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1340cc0f-9bdf-45c3-8be5-5c76d6611e25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06e062f66ada4639b7a8edeaf6f95bf1'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:53.978804 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1340cc0f-9bdf-45c3-8be5-5c76d6611e25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06e062f66ada4639b7a8edeaf6f95bf1'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:53.996203 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1340cc0f-9bdf-45c3-8be5-5c76d6611e25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06e062f66ada4639b7a8edeaf6f95bf1'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:54.010461 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1340cc0f-9bdf-45c3-8be5-5c76d6611e25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06e062f66ada4639b7a8edeaf6f95bf1'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:54.030126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1238/2481] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:44:53 2022] DELETE /identity/v3/users/06e062f66ada4639b7a8edeaf6f95bf1 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:44:54.033966 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c5480-f1dc-467d-a1ec-ebd9f69fe801 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:54.034706 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c5480-f1dc-467d-a1ec-ebd9f69fe801 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:54.034706 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c5480-f1dc-467d-a1ec-ebd9f69fe801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:54.055707 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-497c5480-f1dc-467d-a1ec-ebd9f69fe801 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:44:54.057035 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-497c5480-f1dc-467d-a1ec-ebd9f69fe801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:54.063424 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a196c1-4bff-4f7f-b1cb-b03f8bb40516 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:54.071514 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:44:54.152588 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1239/2482] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:44:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:44:54.156209 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2d2be7-3e3f-42bb-846b-85f3d8dffebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:54.157787 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d2be7-3e3f-42bb-846b-85f3d8dffebd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:54.158018 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d2be7-3e3f-42bb-846b-85f3d8dffebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:54.158340 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d2be7-3e3f-42bb-846b-85f3d8dffebd admin admin] PATH_INFO: `/v3/users/f44d30bba9364527a0db0a7155356cd8` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:54.159728 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2d2be7-3e3f-42bb-846b-85f3d8dffebd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f44d30bba9364527a0db0a7155356cd8)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:54.161892 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2d2be7-3e3f-42bb-846b-85f3d8dffebd admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:54.164554 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c5480-f1dc-467d-a1ec-ebd9f69fe801 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:54.166255 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-497c5480-f1dc-467d-a1ec-ebd9f69fe801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:54.180663 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:44:54.241384 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c5480-f1dc-467d-a1ec-ebd9f69fe801 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:54.242107 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-1a2d2be7-3e3f-42bb-846b-85f3d8dffebd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f44d30bba9364527a0db0a7155356cd8'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:54.243062 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-1a2d2be7-3e3f-42bb-846b-85f3d8dffebd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f44d30bba9364527a0db0a7155356cd8'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:54.245835 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a0327-b28e-4193-aa79-ffc5f404a95c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:54.246632 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a0327-b28e-4193-aa79-ffc5f404a95c service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:54.246922 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a0327-b28e-4193-aa79-ffc5f404a95c service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:54.247443 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a0327-b28e-4193-aa79-ffc5f404a95c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:54.249743 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-aa5a0327-b28e-4193-aa79-ffc5f404a95c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:54.253546 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-1a2d2be7-3e3f-42bb-846b-85f3d8dffebd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f44d30bba9364527a0db0a7155356cd8'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:54.254631 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-1a2d2be7-3e3f-42bb-846b-85f3d8dffebd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f44d30bba9364527a0db0a7155356cd8'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:54.262647 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5a0327-b28e-4193-aa79-ffc5f404a95c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:54.264213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5a0327-b28e-4193-aa79-ffc5f404a95c service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:54.265663 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-1a2d2be7-3e3f-42bb-846b-85f3d8dffebd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f44d30bba9364527a0db0a7155356cd8'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:54.274506 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1240/2483] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:44:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:44:54.294704 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1244/2484] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:44:53 2022] DELETE /identity/v3/users/f44d30bba9364527a0db0a7155356cd8 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:44:54.758811 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a0327-b28e-4193-aa79-ffc5f404a95c service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:54.761184 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-aa5a0327-b28e-4193-aa79-ffc5f404a95c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:54.784083 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a0327-b28e-4193-aa79-ffc5f404a95c service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:54.797146 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2d2be7-3e3f-42bb-846b-85f3d8dffebd admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:54.799394 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-1a2d2be7-3e3f-42bb-846b-85f3d8dffebd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:54.880034 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2137b83-980c-4486-a10a-7a8c40e90ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:54.881447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2137b83-980c-4486-a10a-7a8c40e90ce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:54.882117 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2137b83-980c-4486-a10a-7a8c40e90ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:54.882585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2137b83-980c-4486-a10a-7a8c40e90ce7 admin admin] PATH_INFO: `/v3/projects/1afcb3ba6f8a498d8061ad7ab16e8edc` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:54.884244 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2137b83-980c-4486-a10a-7a8c40e90ce7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1afcb3ba6f8a498d8061ad7ab16e8edc)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:54.886865 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2137b83-980c-4486-a10a-7a8c40e90ce7 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:54.927155 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2d2be7-3e3f-42bb-846b-85f3d8dffebd admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:54.932222 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d3b31-80b0-4717-b437-41bdeafb9ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:54.933129 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d3b31-80b0-4717-b437-41bdeafb9ee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:54.933417 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d3b31-80b0-4717-b437-41bdeafb9ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:54.934262 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d3b31-80b0-4717-b437-41bdeafb9ee5 admin admin] PATH_INFO: `/v3/projects/7c74cd1f44754d9991eec7ecdcfd3b04` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:54.936565 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073d3b31-80b0-4717-b437-41bdeafb9ee5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c74cd1f44754d9991eec7ecdcfd3b04)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:54.938946 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073d3b31-80b0-4717-b437-41bdeafb9ee5 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:54.964912 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f2137b83-980c-4486-a10a-7a8c40e90ce7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1afcb3ba6f8a498d8061ad7ab16e8edc', 'request_id': 'req-f2137b83-980c-4486-a10a-7a8c40e90ce7'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:54.977889 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f2137b83-980c-4486-a10a-7a8c40e90ce7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1afcb3ba6f8a498d8061ad7ab16e8edc', 'request_id': 'req-f2137b83-980c-4486-a10a-7a8c40e90ce7'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:55.000028 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f2137b83-980c-4486-a10a-7a8c40e90ce7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1afcb3ba6f8a498d8061ad7ab16e8edc', 'request_id': 'req-f2137b83-980c-4486-a10a-7a8c40e90ce7'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:55.002567 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:44:55.025153 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f2137b83-980c-4486-a10a-7a8c40e90ce7 admin admin] The token cache is being invalidate because project 1afcb3ba6f8a498d8061ad7ab16e8edc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:44:55.025451 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f2137b83-980c-4486-a10a-7a8c40e90ce7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:55.027847 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f2137b83-980c-4486-a10a-7a8c40e90ce7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:55.030552 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1241/2485] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:44:54 2022] DELETE /identity/v3/projects/1afcb3ba6f8a498d8061ad7ab16e8edc => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:44:55.035211 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f352a-369f-4443-9511-92ae0f66072a None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:55.035455 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-073d3b31-80b0-4717-b437-41bdeafb9ee5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c74cd1f44754d9991eec7ecdcfd3b04', 'request_id': 'req-073d3b31-80b0-4717-b437-41bdeafb9ee5'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:55.035566 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f352a-369f-4443-9511-92ae0f66072a None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:55.035777 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f352a-369f-4443-9511-92ae0f66072a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:55.044355 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-073d3b31-80b0-4717-b437-41bdeafb9ee5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c74cd1f44754d9991eec7ecdcfd3b04', 'request_id': 'req-073d3b31-80b0-4717-b437-41bdeafb9ee5'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:55.052786 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-073d3b31-80b0-4717-b437-41bdeafb9ee5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c74cd1f44754d9991eec7ecdcfd3b04', 'request_id': 'req-073d3b31-80b0-4717-b437-41bdeafb9ee5'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:55.061708 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-da0f352a-369f-4443-9511-92ae0f66072a None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:44:55.063193 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-da0f352a-369f-4443-9511-92ae0f66072a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:55.067849 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-073d3b31-80b0-4717-b437-41bdeafb9ee5 admin admin] The token cache is being invalidate because project 7c74cd1f44754d9991eec7ecdcfd3b04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:44:55.068151 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-073d3b31-80b0-4717-b437-41bdeafb9ee5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:55.068854 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-073d3b31-80b0-4717-b437-41bdeafb9ee5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:44:55.070790 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1245/2486] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:44:54 2022] DELETE /identity/v3/projects/7c74cd1f44754d9991eec7ecdcfd3b04 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:44:55.078145 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043aa246-f43d-4e93-ae83-8ca63bda397d None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:55.078145 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043aa246-f43d-4e93-ae83-8ca63bda397d None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:55.078574 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043aa246-f43d-4e93-ae83-8ca63bda397d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:55.104507 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-043aa246-f43d-4e93-ae83-8ca63bda397d None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:44:55.106054 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-043aa246-f43d-4e93-ae83-8ca63bda397d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:55.126079 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:44:55.151713 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1242/2487] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:44:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:44:55.160455 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298e1d1-1d26-440b-afcb-3abb6ae9aa9b None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:55.161788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298e1d1-1d26-440b-afcb-3abb6ae9aa9b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:55.162071 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298e1d1-1d26-440b-afcb-3abb6ae9aa9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:55.181760 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-a298e1d1-1d26-440b-afcb-3abb6ae9aa9b None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:44:55.183043 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a298e1d1-1d26-440b-afcb-3abb6ae9aa9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:55.191834 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1243/2488] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:44:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:44:55.193862 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1246/2489] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:44:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:44:55.196171 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a298e1d1-1d26-440b-afcb-3abb6ae9aa9b None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:55.202580 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a8098-f50e-4fd1-aa71-19cbfb74b027 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:55.202850 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a8098-f50e-4fd1-aa71-19cbfb74b027 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:55.203296 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a8098-f50e-4fd1-aa71-19cbfb74b027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:55.206867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a298e1d1-1d26-440b-afcb-3abb6ae9aa9b None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:55.213734 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf2d03-eba1-4544-b68e-f329f6479389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:55.214459 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf2d03-eba1-4544-b68e-f329f6479389 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:55.214743 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf2d03-eba1-4544-b68e-f329f6479389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:55.215018 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf2d03-eba1-4544-b68e-f329f6479389 admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:55.215760 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daf2d03-eba1-4544-b68e-f329f6479389 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:44:55.216067 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daf2d03-eba1-4544-b68e-f329f6479389 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:55.217767 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daf2d03-eba1-4544-b68e-f329f6479389 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:55.224395 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-525a8098-f50e-4fd1-aa71-19cbfb74b027 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:44:55.225641 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-525a8098-f50e-4fd1-aa71-19cbfb74b027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:55.230281 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1244/2490] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:44:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:44:55.235049 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf2d03-eba1-4544-b68e-f329f6479389 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:55.238214 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1247/2491] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:44:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:44:55.240821 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a8098-f50e-4fd1-aa71-19cbfb74b027 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:55.245511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf2d03-eba1-4544-b68e-f329f6479389 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:55.251087 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e59bc-7092-4dd4-8f71-478cf131392b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:55.251765 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e59bc-7092-4dd4-8f71-478cf131392b admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:55.252179 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e59bc-7092-4dd4-8f71-478cf131392b admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:55.252612 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e59bc-7092-4dd4-8f71-478cf131392b admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:55.253164 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e59bc-7092-4dd4-8f71-478cf131392b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:55.253515 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a8098-f50e-4fd1-aa71-19cbfb74b027 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:55.255226 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e59bc-7092-4dd4-8f71-478cf131392b admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:55.259479 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8fbfb4-0fea-4717-b859-7c8b25026d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:55.260350 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8fbfb4-0fea-4717-b859-7c8b25026d98 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:55.260542 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8fbfb4-0fea-4717-b859-7c8b25026d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:55.260824 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8fbfb4-0fea-4717-b859-7c8b25026d98 admin admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:55.261594 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8fbfb4-0fea-4717-b859-7c8b25026d98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:44:55.261882 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8fbfb4-0fea-4717-b859-7c8b25026d98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:55.263564 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8fbfb4-0fea-4717-b859-7c8b25026d98 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:55.284581 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1248/2492] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:44:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:44:55.288529 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8fbfb4-0fea-4717-b859-7c8b25026d98 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:55.298386 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1245/2493] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:44:55 2022] POST /identity/v3/projects => generated 371 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:44:55.299359 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e59bc-7092-4dd4-8f71-478cf131392b admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:55.309706 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8fbfb4-0fea-4717-b859-7c8b25026d98 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:55.406951 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991806f4-53e0-43c2-9a0e-773370a910f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:55.407978 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991806f4-53e0-43c2-9a0e-773370a910f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:55.408925 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991806f4-53e0-43c2-9a0e-773370a910f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:55.409344 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991806f4-53e0-43c2-9a0e-773370a910f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:55.410357 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991806f4-53e0-43c2-9a0e-773370a910f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:55.413021 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991806f4-53e0-43c2-9a0e-773370a910f3 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:55.431182 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e59bc-7092-4dd4-8f71-478cf131392b admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:55.448180 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0af3f4-096f-4520-9027-6f0f9cf75a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:55.449478 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0af3f4-096f-4520-9027-6f0f9cf75a75 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:55.450049 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0af3f4-096f-4520-9027-6f0f9cf75a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:55.450578 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0af3f4-096f-4520-9027-6f0f9cf75a75 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:55.452358 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0af3f4-096f-4520-9027-6f0f9cf75a75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:55.457691 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0af3f4-096f-4520-9027-6f0f9cf75a75 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:55.472540 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1249/2494] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:44:55 2022] POST /identity/v3/projects => generated 383 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:44:55.474219 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991806f4-53e0-43c2-9a0e-773370a910f3 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:55.506303 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1246/2495] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:44:55 2022] POST /identity/v3/users => generated 286 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:44:55.514581 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0af3f4-096f-4520-9027-6f0f9cf75a75 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:55.535536 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0af3f4-096f-4520-9027-6f0f9cf75a75 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:55.560530 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991806f4-53e0-43c2-9a0e-773370a910f3 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:55.566386 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637cd756-bc03-40c1-8ca7-325d2423990b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:55.567098 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637cd756-bc03-40c1-8ca7-325d2423990b admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:55.567443 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637cd756-bc03-40c1-8ca7-325d2423990b admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:55.567727 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637cd756-bc03-40c1-8ca7-325d2423990b admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:55.568493 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637cd756-bc03-40c1-8ca7-325d2423990b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:55.570707 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637cd756-bc03-40c1-8ca7-325d2423990b admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:55.588461 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:44:55.606618 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1250/2496] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:44:55 2022] POST /identity/v3/users => generated 292 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:44:55.613044 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637cd756-bc03-40c1-8ca7-325d2423990b admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:55.620879 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae87466-a808-4ab7-8034-48dabad0f3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:55.621582 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae87466-a808-4ab7-8034-48dabad0f3c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:55.621929 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae87466-a808-4ab7-8034-48dabad0f3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:55.622261 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae87466-a808-4ab7-8034-48dabad0f3c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:55.623653 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae87466-a808-4ab7-8034-48dabad0f3c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:55.625377 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae87466-a808-4ab7-8034-48dabad0f3c6 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:55.625823 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637cd756-bc03-40c1-8ca7-325d2423990b admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:55.635702 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1456b9e8-63ac-49c1-971c-4766335f959c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:55.636643 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456b9e8-63ac-49c1-971c-4766335f959c admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:55.636939 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456b9e8-63ac-49c1-971c-4766335f959c admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:55.637220 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456b9e8-63ac-49c1-971c-4766335f959c admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:55.638175 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1456b9e8-63ac-49c1-971c-4766335f959c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:55.639982 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1456b9e8-63ac-49c1-971c-4766335f959c admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:55.649647 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1247/2497] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:44:55.653990 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae87466-a808-4ab7-8034-48dabad0f3c6 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:55.661343 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1251/2498] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:44:55.667217 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1456b9e8-63ac-49c1-971c-4766335f959c admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:55.668308 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae87466-a808-4ab7-8034-48dabad0f3c6 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:55.674329 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6035561d-b82f-435a-95d7-2b8f9cff2f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:55.675035 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6035561d-b82f-435a-95d7-2b8f9cff2f9d admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:55.675324 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6035561d-b82f-435a-95d7-2b8f9cff2f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:55.675602 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6035561d-b82f-435a-95d7-2b8f9cff2f9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:55.676574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6035561d-b82f-435a-95d7-2b8f9cff2f9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:55.678286 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6035561d-b82f-435a-95d7-2b8f9cff2f9d admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:55.680949 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:44:55.691213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1248/2499] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:44:55.692052 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1456b9e8-63ac-49c1-971c-4766335f959c admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:55.695199 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6035561d-b82f-435a-95d7-2b8f9cff2f9d admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:55.705125 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd042b92-e220-48fb-8360-e1f140a163f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:55.706371 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6035561d-b82f-435a-95d7-2b8f9cff2f9d admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:55.706659 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd042b92-e220-48fb-8360-e1f140a163f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:55.707524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd042b92-e220-48fb-8360-e1f140a163f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:55.708178 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd042b92-e220-48fb-8360-e1f140a163f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:55.709666 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd042b92-e220-48fb-8360-e1f140a163f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:55.711509 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb06d78e-26f3-4295-853b-296426a09c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:55.712478 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06d78e-26f3-4295-853b-296426a09c68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:55.712835 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06d78e-26f3-4295-853b-296426a09c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:55.713134 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd042b92-e220-48fb-8360-e1f140a163f7 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:55.713253 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06d78e-26f3-4295-853b-296426a09c68 admin admin] PATH_INFO: `/v3/projects/cbaf2f777f01437ca5891860365e08ce/users/156daafe58fd4e248020b225b9c2c210/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:55.730825 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1252/2500] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:44:55.737533 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd042b92-e220-48fb-8360-e1f140a163f7 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:55.738262 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb06d78e-26f3-4295-853b-296426a09c68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbaf2f777f01437ca5891860365e08ce, user_id=156daafe58fd4e248020b225b9c2c210, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:55.740989 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb06d78e-26f3-4295-853b-296426a09c68 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:55.751489 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd042b92-e220-48fb-8360-e1f140a163f7 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:55.763386 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1249/2501] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:44:55 2022] PUT /identity/v3/projects/cbaf2f777f01437ca5891860365e08ce/users/156daafe58fd4e248020b225b9c2c210/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:44:55.769088 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca9b95-96a1-47fc-a098-7ec826188fc0 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:55.769492 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca9b95-96a1-47fc-a098-7ec826188fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:55.769870 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca9b95-96a1-47fc-a098-7ec826188fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:55.788361 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-aaca9b95-96a1-47fc-a098-7ec826188fc0 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:44:55.789734 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-aaca9b95-96a1-47fc-a098-7ec826188fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:55.840134 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7f8a9-d260-4b8d-9410-d99056e45afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:55.842051 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7f8a9-d260-4b8d-9410-d99056e45afd admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:55.842806 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7f8a9-d260-4b8d-9410-d99056e45afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:55.843099 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7f8a9-d260-4b8d-9410-d99056e45afd admin admin] PATH_INFO: `/v3/projects/1f5fdc2e46a5465da90cd64d65543704/users/cac7d4d9659d4a048ef5129d69b67831/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:55.864707 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea7f8a9-d260-4b8d-9410-d99056e45afd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f5fdc2e46a5465da90cd64d65543704, user_id=cac7d4d9659d4a048ef5129d69b67831, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:55.868277 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea7f8a9-d260-4b8d-9410-d99056e45afd admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:55.885499 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1253/2502] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:44:55 2022] PUT /identity/v3/projects/1f5fdc2e46a5465da90cd64d65543704/users/cac7d4d9659d4a048ef5129d69b67831/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:44:55.896403 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1250/2503] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:44:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:44:55.899616 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f680766-b793-4080-b9e8-9aa7140b7a58 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:55.899616 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f680766-b793-4080-b9e8-9aa7140b7a58 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:55.899616 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f680766-b793-4080-b9e8-9aa7140b7a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:55.907369 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca9b95-96a1-47fc-a098-7ec826188fc0 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:55.909312 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-aaca9b95-96a1-47fc-a098-7ec826188fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:55.930157 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-9f680766-b793-4080-b9e8-9aa7140b7a58 None None] MFA Rules not processed for user `cac7d4d9659d4a048ef5129d69b67831`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:44:55.931725 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-9f680766-b793-4080-b9e8-9aa7140b7a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:55.984216 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca9b95-96a1-47fc-a098-7ec826188fc0 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:56.022663 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1254/2504] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:44:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:44:56.033198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f680766-b793-4080-b9e8-9aa7140b7a58 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:56.036566 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-9f680766-b793-4080-b9e8-9aa7140b7a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:56.051709 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b073576-9f12-4851-97b7-6f017fa699b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:56.052907 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b073576-9f12-4851-97b7-6f017fa699b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:56.053441 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b073576-9f12-4851-97b7-6f017fa699b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:56.053978 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b073576-9f12-4851-97b7-6f017fa699b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:56.072885 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:44:56.125045 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f680766-b793-4080-b9e8-9aa7140b7a58 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:56.129866 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480bba39-c874-4f89-be57-e97c35aa4078 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:56.130831 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480bba39-c874-4f89-be57-e97c35aa4078 service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:56.131424 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480bba39-c874-4f89-be57-e97c35aa4078 service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:56.131707 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480bba39-c874-4f89-be57-e97c35aa4078 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:56.153085 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480bba39-c874-4f89-be57-e97c35aa4078 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:56.156116 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480bba39-c874-4f89-be57-e97c35aa4078 service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:56.174520 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1255/2505] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:44:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:44:56.180394 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b073576-9f12-4851-97b7-6f017fa699b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:56.182113 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b073576-9f12-4851-97b7-6f017fa699b8 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:56.197120 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1251/2506] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:44:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:44:57.518465 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f654f-f6e9-494a-a1c7-4100ea7c916f None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:57.519140 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f654f-f6e9-494a-a1c7-4100ea7c916f None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:57.519297 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f654f-f6e9-494a-a1c7-4100ea7c916f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:57.550011 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:44:57.562072 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-e04f654f-f6e9-494a-a1c7-4100ea7c916f None None] MFA Rules not processed for user `156daafe58fd4e248020b225b9c2c210`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:44:57.564082 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-e04f654f-f6e9-494a-a1c7-4100ea7c916f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:57.664682 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1256/2507] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:44:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:44:57.669323 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b073576-9f12-4851-97b7-6f017fa699b8 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:57.680131 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b073576-9f12-4851-97b7-6f017fa699b8 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:57.685460 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e489c20a-490b-4d71-80be-726305123a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:57.686737 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489c20a-490b-4d71-80be-726305123a64 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:57.687032 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489c20a-490b-4d71-80be-726305123a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:57.687314 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489c20a-490b-4d71-80be-726305123a64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:57.688052 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e489c20a-490b-4d71-80be-726305123a64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:57.690143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e489c20a-490b-4d71-80be-726305123a64 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:57.730644 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1252/2508] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:44:57 2022] POST /identity/v3/projects => generated 369 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:44:57.735000 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f654f-f6e9-494a-a1c7-4100ea7c916f None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:57.754414 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f654f-f6e9-494a-a1c7-4100ea7c916f None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:57.917501 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a1bcf7-1f41-47bb-95c2-9c9c21f2e6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:57.921308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1bcf7-1f41-47bb-95c2-9c9c21f2e6ff admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:57.922172 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1bcf7-1f41-47bb-95c2-9c9c21f2e6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:57.922497 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1bcf7-1f41-47bb-95c2-9c9c21f2e6ff admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:57.925028 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a1bcf7-1f41-47bb-95c2-9c9c21f2e6ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:57.931027 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a1bcf7-1f41-47bb-95c2-9c9c21f2e6ff admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:57.982179 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1257/2509] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:44:57 2022] POST /identity/v3/users => generated 291 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:44:57.985214 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e489c20a-490b-4d71-80be-726305123a64 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:58.001437 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e489c20a-490b-4d71-80be-726305123a64 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:58.007951 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfeb1b9-3219-418d-9762-4354689c4a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:58.008966 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfeb1b9-3219-418d-9762-4354689c4a7f admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:58.009252 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfeb1b9-3219-418d-9762-4354689c4a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:58.009563 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfeb1b9-3219-418d-9762-4354689c4a7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:58.010350 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfeb1b9-3219-418d-9762-4354689c4a7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:58.011987 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfeb1b9-3219-418d-9762-4354689c4a7f admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:58.026579 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1253/2510] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:44:58.030680 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a1bcf7-1f41-47bb-95c2-9c9c21f2e6ff admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:58.043023 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a1bcf7-1f41-47bb-95c2-9c9c21f2e6ff admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:58.048843 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed676a-3617-4677-9e14-ba90c9ac0053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:58.049965 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed676a-3617-4677-9e14-ba90c9ac0053 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:58.050345 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed676a-3617-4677-9e14-ba90c9ac0053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:58.050524 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed676a-3617-4677-9e14-ba90c9ac0053 admin admin] PATH_INFO: `/v3/projects/b2d031f1a6354c9a9722aefa5cfe72f4/users/ccf4b7d559644c6887dbeac50812df74/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:58.075751 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ed676a-3617-4677-9e14-ba90c9ac0053 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2d031f1a6354c9a9722aefa5cfe72f4, user_id=ccf4b7d559644c6887dbeac50812df74, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:58.080525 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ed676a-3617-4677-9e14-ba90c9ac0053 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:58.109129 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1258/2511] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:44:58 2022] PUT /identity/v3/projects/b2d031f1a6354c9a9722aefa5cfe72f4/users/ccf4b7d559644c6887dbeac50812df74/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:44:58.113198 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfeb1b9-3219-418d-9762-4354689c4a7f admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:58.125977 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfeb1b9-3219-418d-9762-4354689c4a7f admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:58.199515 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a22a6ce-9af7-4352-a9b4-71becfeee1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:58.200304 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22a6ce-9af7-4352-a9b4-71becfeee1ba admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:58.200583 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22a6ce-9af7-4352-a9b4-71becfeee1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:58.200859 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22a6ce-9af7-4352-a9b4-71becfeee1ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:58.202310 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a22a6ce-9af7-4352-a9b4-71becfeee1ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:58.204316 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a22a6ce-9af7-4352-a9b4-71becfeee1ba admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:58.216098 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1254/2512] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:44:58 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:44:58.221560 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed676a-3617-4677-9e14-ba90c9ac0053 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:58.236153 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed676a-3617-4677-9e14-ba90c9ac0053 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:58.241210 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87ee29-9f70-4946-8b84-c5f31ce5f152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:58.242175 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87ee29-9f70-4946-8b84-c5f31ce5f152 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:58.242880 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87ee29-9f70-4946-8b84-c5f31ce5f152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:58.243169 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87ee29-9f70-4946-8b84-c5f31ce5f152 admin admin] PATH_INFO: `/v3/projects/b2d031f1a6354c9a9722aefa5cfe72f4/users/ccf4b7d559644c6887dbeac50812df74/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:58.245288 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a87ee29-9f70-4946-8b84-c5f31ce5f152 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2d031f1a6354c9a9722aefa5cfe72f4, user_id=ccf4b7d559644c6887dbeac50812df74, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:58.248080 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a87ee29-9f70-4946-8b84-c5f31ce5f152 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:58.260424 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1259/2513] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:44:58 2022] PUT /identity/v3/projects/b2d031f1a6354c9a9722aefa5cfe72f4/users/ccf4b7d559644c6887dbeac50812df74/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:44:59.115093 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7bb92-2530-484d-b150-4e7bbbd037a8 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:59.115991 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7bb92-2530-484d-b150-4e7bbbd037a8 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:59.116587 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7bb92-2530-484d-b150-4e7bbbd037a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:59.149868 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-edc7bb92-2530-484d-b150-4e7bbbd037a8 None None] MFA Rules not processed for user `ccf4b7d559644c6887dbeac50812df74`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:44:59.151409 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-edc7bb92-2530-484d-b150-4e7bbbd037a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:44:59.237979 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:44:59.258418 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1255/2514] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:44:59 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:44:59.271846 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87ee29-9f70-4946-8b84-c5f31ce5f152 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:44:59.361815 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87ee29-9f70-4946-8b84-c5f31ce5f152 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:44:59.365869 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c9f6b5-040a-45f1-b36b-b277134e3031 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:44:59.367296 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c9f6b5-040a-45f1-b36b-b277134e3031 service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:44:59.367296 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c9f6b5-040a-45f1-b36b-b277134e3031 service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:44:59.367948 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c9f6b5-040a-45f1-b36b-b277134e3031 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:44:59.378074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:44:59.462678 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c9f6b5-040a-45f1-b36b-b277134e3031 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:44:59.464233 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c9f6b5-040a-45f1-b36b-b277134e3031 service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:44:59.473802 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1260/2515] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:44:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:45:01.480647 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7bb92-2530-484d-b150-4e7bbbd037a8 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:01.484094 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-edc7bb92-2530-484d-b150-4e7bbbd037a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:01.552545 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7bb92-2530-484d-b150-4e7bbbd037a8 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:01.559365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84781244-17c1-4b8c-a06d-793511162b36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:01.559365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84781244-17c1-4b8c-a06d-793511162b36 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:01.559365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84781244-17c1-4b8c-a06d-793511162b36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:01.559365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84781244-17c1-4b8c-a06d-793511162b36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:01.560999 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-84781244-17c1-4b8c-a06d-793511162b36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:01.650120 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84781244-17c1-4b8c-a06d-793511162b36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:01.652563 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84781244-17c1-4b8c-a06d-793511162b36 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:01.668717 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1256/2516] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:45:01 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:45:01.995627 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c9f6b5-040a-45f1-b36b-b277134e3031 service cinder] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:02.005575 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c9f6b5-040a-45f1-b36b-b277134e3031 service cinder] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:02.009513 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed820a58-ef6e-4992-934b-81e879826478 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:02.010213 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed820a58-ef6e-4992-934b-81e879826478 service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:02.010504 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed820a58-ef6e-4992-934b-81e879826478 service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:02.010791 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed820a58-ef6e-4992-934b-81e879826478 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:02.021216 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed820a58-ef6e-4992-934b-81e879826478 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:02.022926 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed820a58-ef6e-4992-934b-81e879826478 service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:02.033686 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1261/2517] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:45:01 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:45:03.492754 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84781244-17c1-4b8c-a06d-793511162b36 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:03.508035 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84781244-17c1-4b8c-a06d-793511162b36 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:03.514664 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff79ae98-cf3f-47ba-aafe-d171c827925e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:03.515366 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79ae98-cf3f-47ba-aafe-d171c827925e service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:03.515765 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79ae98-cf3f-47ba-aafe-d171c827925e service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:03.516243 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79ae98-cf3f-47ba-aafe-d171c827925e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:03.518250 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ff79ae98-cf3f-47ba-aafe-d171c827925e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:03.619130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff79ae98-cf3f-47ba-aafe-d171c827925e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:03.620664 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff79ae98-cf3f-47ba-aafe-d171c827925e service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:03.647810 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1257/2518] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:45:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:45:04.093028 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed820a58-ef6e-4992-934b-81e879826478 service cinder] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:04.108873 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed820a58-ef6e-4992-934b-81e879826478 service cinder] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:04.117005 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4fa86-1ecb-4d3f-953a-9120effdacea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:04.118368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4fa86-1ecb-4d3f-953a-9120effdacea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:04.118757 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4fa86-1ecb-4d3f-953a-9120effdacea admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:04.119138 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4fa86-1ecb-4d3f-953a-9120effdacea admin admin] PATH_INFO: `/v3/users/cac7d4d9659d4a048ef5129d69b67831` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:04.120665 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b4fa86-1ecb-4d3f-953a-9120effdacea admin admin] RBAC: Authorizing `identity:delete_user(user_id=cac7d4d9659d4a048ef5129d69b67831)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:04.125828 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b4fa86-1ecb-4d3f-953a-9120effdacea admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:04.189482 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-65b4fa86-1ecb-4d3f-953a-9120effdacea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cac7d4d9659d4a048ef5129d69b67831'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:04.190500 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-65b4fa86-1ecb-4d3f-953a-9120effdacea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cac7d4d9659d4a048ef5129d69b67831'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:04.200289 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-65b4fa86-1ecb-4d3f-953a-9120effdacea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cac7d4d9659d4a048ef5129d69b67831'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:04.201064 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-65b4fa86-1ecb-4d3f-953a-9120effdacea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cac7d4d9659d4a048ef5129d69b67831'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:04.214560 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-65b4fa86-1ecb-4d3f-953a-9120effdacea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cac7d4d9659d4a048ef5129d69b67831'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:04.254562 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1262/2519] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:45:04 2022] DELETE /identity/v3/users/cac7d4d9659d4a048ef5129d69b67831 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:45:04.259198 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c489d12-ad12-4f98-a57f-34b18f8fdc46 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:04.259500 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c489d12-ad12-4f98-a57f-34b18f8fdc46 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:04.259778 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c489d12-ad12-4f98-a57f-34b18f8fdc46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:04.280907 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-2c489d12-ad12-4f98-a57f-34b18f8fdc46 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:45:04.282381 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-2c489d12-ad12-4f98-a57f-34b18f8fdc46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:04.324692 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:04.362865 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1258/2520] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:45:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:45:04.375233 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4fa86-1ecb-4d3f-953a-9120effdacea admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:04.377477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-65b4fa86-1ecb-4d3f-953a-9120effdacea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:04.453464 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4fa86-1ecb-4d3f-953a-9120effdacea admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:04.457448 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e816a4b-e322-46f0-94c2-aa3c5bc3cf75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:04.458154 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e816a4b-e322-46f0-94c2-aa3c5bc3cf75 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:04.458499 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e816a4b-e322-46f0-94c2-aa3c5bc3cf75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:04.458783 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e816a4b-e322-46f0-94c2-aa3c5bc3cf75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:04.471035 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e816a4b-e322-46f0-94c2-aa3c5bc3cf75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:04.472595 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e816a4b-e322-46f0-94c2-aa3c5bc3cf75 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:04.488588 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1263/2521] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:45:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:45:04.829975 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c489d12-ad12-4f98-a57f-34b18f8fdc46 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:04.831910 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-2c489d12-ad12-4f98-a57f-34b18f8fdc46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:04.863614 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c489d12-ad12-4f98-a57f-34b18f8fdc46 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:04.873832 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e457b-0e53-4508-8aa2-4d8b89102a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:04.875181 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e457b-0e53-4508-8aa2-4d8b89102a96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:04.875906 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e457b-0e53-4508-8aa2-4d8b89102a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:04.876352 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e457b-0e53-4508-8aa2-4d8b89102a96 admin admin] PATH_INFO: `/v3/projects/1f5fdc2e46a5465da90cd64d65543704` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:04.878463 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e457b-0e53-4508-8aa2-4d8b89102a96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f5fdc2e46a5465da90cd64d65543704)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:04.880796 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e457b-0e53-4508-8aa2-4d8b89102a96 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:04.967825 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-417e457b-0e53-4508-8aa2-4d8b89102a96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f5fdc2e46a5465da90cd64d65543704', 'request_id': 'req-417e457b-0e53-4508-8aa2-4d8b89102a96'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:04.985196 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-417e457b-0e53-4508-8aa2-4d8b89102a96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f5fdc2e46a5465da90cd64d65543704', 'request_id': 'req-417e457b-0e53-4508-8aa2-4d8b89102a96'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:04.994706 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-417e457b-0e53-4508-8aa2-4d8b89102a96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f5fdc2e46a5465da90cd64d65543704', 'request_id': 'req-417e457b-0e53-4508-8aa2-4d8b89102a96'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:05.011077 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-417e457b-0e53-4508-8aa2-4d8b89102a96 admin admin] The token cache is being invalidate because project 1f5fdc2e46a5465da90cd64d65543704 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:45:05.011353 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-417e457b-0e53-4508-8aa2-4d8b89102a96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:05.012752 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-417e457b-0e53-4508-8aa2-4d8b89102a96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:05.018871 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1259/2522] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:45:04 2022] DELETE /identity/v3/projects/1f5fdc2e46a5465da90cd64d65543704 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:45:05.021577 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd6b0c-f315-4659-9238-70ed41681cd5 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:05.022211 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd6b0c-f315-4659-9238-70ed41681cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:05.022393 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd6b0c-f315-4659-9238-70ed41681cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:05.035928 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:05.044088 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-16dd6b0c-f315-4659-9238-70ed41681cd5 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:45:05.046944 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-16dd6b0c-f315-4659-9238-70ed41681cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:05.151956 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1264/2523] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:45:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:45:05.165341 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6735f2-1bce-4910-b69b-4c8857e2cfbd None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:05.165720 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6735f2-1bce-4910-b69b-4c8857e2cfbd None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:05.166071 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6735f2-1bce-4910-b69b-4c8857e2cfbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:05.186349 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-1a6735f2-1bce-4910-b69b-4c8857e2cfbd None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:45:05.187627 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1a6735f2-1bce-4910-b69b-4c8857e2cfbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:05.198148 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1260/2524] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:45:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:45:05.201864 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dd6b0c-f315-4659-9238-70ed41681cd5 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:05.214320 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dd6b0c-f315-4659-9238-70ed41681cd5 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:05.219993 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89a752-7e4d-4aac-b7b6-6cf736238ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:05.220684 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89a752-7e4d-4aac-b7b6-6cf736238ab6 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:05.221132 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89a752-7e4d-4aac-b7b6-6cf736238ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:05.221413 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89a752-7e4d-4aac-b7b6-6cf736238ab6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:05.222228 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a89a752-7e4d-4aac-b7b6-6cf736238ab6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:45:05.222548 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a89a752-7e4d-4aac-b7b6-6cf736238ab6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:05.224182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a89a752-7e4d-4aac-b7b6-6cf736238ab6 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:05.238328 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1265/2525] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:45:05.243166 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6735f2-1bce-4910-b69b-4c8857e2cfbd None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:05.253205 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6735f2-1bce-4910-b69b-4c8857e2cfbd None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:05.258330 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f7664a-bcf4-4e7a-8db6-79469b455b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:05.259020 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7664a-bcf4-4e7a-8db6-79469b455b3a admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:05.259314 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7664a-bcf4-4e7a-8db6-79469b455b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:05.259599 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7664a-bcf4-4e7a-8db6-79469b455b3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:05.260308 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f7664a-bcf4-4e7a-8db6-79469b455b3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:05.262288 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f7664a-bcf4-4e7a-8db6-79469b455b3a admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:05.298839 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1261/2526] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:45:05 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:45:05.303267 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89a752-7e4d-4aac-b7b6-6cf736238ab6 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:05.316765 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89a752-7e4d-4aac-b7b6-6cf736238ab6 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:05.385169 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a62a4-9e27-4027-8466-b5403bd1650a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:05.386064 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a62a4-9e27-4027-8466-b5403bd1650a admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:05.386385 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a62a4-9e27-4027-8466-b5403bd1650a admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:05.386665 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a62a4-9e27-4027-8466-b5403bd1650a admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:05.387366 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a62a4-9e27-4027-8466-b5403bd1650a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:05.389315 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a62a4-9e27-4027-8466-b5403bd1650a admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:05.410001 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1266/2527] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:45:05 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:45:05.414069 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f7664a-bcf4-4e7a-8db6-79469b455b3a admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:05.424970 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f7664a-bcf4-4e7a-8db6-79469b455b3a admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:05.429910 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7db7c2-c70d-4d9c-aa01-5144c34c7547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:05.430701 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7db7c2-c70d-4d9c-aa01-5144c34c7547 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:05.431448 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7db7c2-c70d-4d9c-aa01-5144c34c7547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:05.431448 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7db7c2-c70d-4d9c-aa01-5144c34c7547 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:05.432089 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7db7c2-c70d-4d9c-aa01-5144c34c7547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:05.433797 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7db7c2-c70d-4d9c-aa01-5144c34c7547 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:05.446721 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1262/2528] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:45:05.450655 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a62a4-9e27-4027-8466-b5403bd1650a admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:05.461049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a62a4-9e27-4027-8466-b5403bd1650a admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:05.465953 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf812ea-5b8b-4684-944f-acf1e2a8cc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:05.467348 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf812ea-5b8b-4684-944f-acf1e2a8cc08 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:05.467723 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf812ea-5b8b-4684-944f-acf1e2a8cc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:05.467998 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf812ea-5b8b-4684-944f-acf1e2a8cc08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:05.468731 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf812ea-5b8b-4684-944f-acf1e2a8cc08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:05.470413 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf812ea-5b8b-4684-944f-acf1e2a8cc08 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:05.486253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1267/2529] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:45:05.488040 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7db7c2-c70d-4d9c-aa01-5144c34c7547 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:05.499123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7db7c2-c70d-4d9c-aa01-5144c34c7547 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:05.504202 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71fbcf-ce3d-4339-bd1f-32df0518f2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:05.505110 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71fbcf-ce3d-4339-bd1f-32df0518f2ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:05.505396 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71fbcf-ce3d-4339-bd1f-32df0518f2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:05.505683 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71fbcf-ce3d-4339-bd1f-32df0518f2ff admin admin] PATH_INFO: `/v3/projects/a2d3499fcfd34f3dac6ab8f3e31eea1e/users/ca02102f415b41ae9263dc76a288ce41/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:05.523538 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de71fbcf-ce3d-4339-bd1f-32df0518f2ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2d3499fcfd34f3dac6ab8f3e31eea1e, user_id=ca02102f415b41ae9263dc76a288ce41, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:05.526248 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de71fbcf-ce3d-4339-bd1f-32df0518f2ff admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:05.543791 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1263/2530] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:45:05 2022] PUT /identity/v3/projects/a2d3499fcfd34f3dac6ab8f3e31eea1e/users/ca02102f415b41ae9263dc76a288ce41/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:45:05.555126 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee509c8-2fbe-473b-aca4-69308ab3378a None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:05.555422 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee509c8-2fbe-473b-aca4-69308ab3378a None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:05.555775 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee509c8-2fbe-473b-aca4-69308ab3378a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:05.585185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-0ee509c8-2fbe-473b-aca4-69308ab3378a None None] MFA Rules not processed for user `ca02102f415b41ae9263dc76a288ce41`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:45:05.586736 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-0ee509c8-2fbe-473b-aca4-69308ab3378a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:05.661431 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1268/2531] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:45:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:45:05.673098 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71fbcf-ce3d-4339-bd1f-32df0518f2ff admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:05.674912 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-de71fbcf-ce3d-4339-bd1f-32df0518f2ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:05.735766 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71fbcf-ce3d-4339-bd1f-32df0518f2ff admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:05.739438 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac7de0-a161-4371-9e70-6813ef218db6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:05.740092 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac7de0-a161-4371-9e70-6813ef218db6 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:05.740379 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac7de0-a161-4371-9e70-6813ef218db6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:05.740658 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac7de0-a161-4371-9e70-6813ef218db6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:05.750570 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cac7de0-a161-4371-9e70-6813ef218db6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:05.752040 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cac7de0-a161-4371-9e70-6813ef218db6 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:05.757820 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:05.763740 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1264/2532] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:45:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:45:07.726337 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee509c8-2fbe-473b-aca4-69308ab3378a None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:07.728791 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-0ee509c8-2fbe-473b-aca4-69308ab3378a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:07.735654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:07.838830 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee509c8-2fbe-473b-aca4-69308ab3378a None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:07.844801 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae2b772-9c1b-4459-931d-c111cc993d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:07.845818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2b772-9c1b-4459-931d-c111cc993d5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:07.846282 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2b772-9c1b-4459-931d-c111cc993d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:07.846591 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2b772-9c1b-4459-931d-c111cc993d5f admin admin] PATH_INFO: `/v3/users/b2103a7ed6824444a60fa5573e7ad39e` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:07.853816 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae2b772-9c1b-4459-931d-c111cc993d5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2103a7ed6824444a60fa5573e7ad39e)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:07.856090 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae2b772-9c1b-4459-931d-c111cc993d5f admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:07.923431 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-2ae2b772-9c1b-4459-931d-c111cc993d5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2103a7ed6824444a60fa5573e7ad39e'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:07.924120 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-2ae2b772-9c1b-4459-931d-c111cc993d5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2103a7ed6824444a60fa5573e7ad39e'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:07.933347 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-2ae2b772-9c1b-4459-931d-c111cc993d5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2103a7ed6824444a60fa5573e7ad39e'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:07.933989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-2ae2b772-9c1b-4459-931d-c111cc993d5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2103a7ed6824444a60fa5573e7ad39e'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:07.946340 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-2ae2b772-9c1b-4459-931d-c111cc993d5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2103a7ed6824444a60fa5573e7ad39e'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:07.964078 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1269/2533] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:45:07 2022] DELETE /identity/v3/users/b2103a7ed6824444a60fa5573e7ad39e => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:45:08.148570 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac7de0-a161-4371-9e70-6813ef218db6 service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:08.164613 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac7de0-a161-4371-9e70-6813ef218db6 service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:08.221514 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48587c01-6994-4938-ad2c-336a5863cbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:08.222427 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48587c01-6994-4938-ad2c-336a5863cbb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:08.222708 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48587c01-6994-4938-ad2c-336a5863cbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:08.222984 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48587c01-6994-4938-ad2c-336a5863cbb5 admin admin] PATH_INFO: `/v3/projects/71e0c71509f94f158063991e37617894` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:08.224187 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48587c01-6994-4938-ad2c-336a5863cbb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71e0c71509f94f158063991e37617894)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:08.226077 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48587c01-6994-4938-ad2c-336a5863cbb5 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:08.271642 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-48587c01-6994-4938-ad2c-336a5863cbb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71e0c71509f94f158063991e37617894', 'request_id': 'req-48587c01-6994-4938-ad2c-336a5863cbb5'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:08.280390 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-48587c01-6994-4938-ad2c-336a5863cbb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71e0c71509f94f158063991e37617894', 'request_id': 'req-48587c01-6994-4938-ad2c-336a5863cbb5'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:08.286693 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-48587c01-6994-4938-ad2c-336a5863cbb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71e0c71509f94f158063991e37617894', 'request_id': 'req-48587c01-6994-4938-ad2c-336a5863cbb5'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:08.297612 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-48587c01-6994-4938-ad2c-336a5863cbb5 admin admin] The token cache is being invalidate because project 71e0c71509f94f158063991e37617894 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:45:08.297899 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-48587c01-6994-4938-ad2c-336a5863cbb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:08.298589 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-48587c01-6994-4938-ad2c-336a5863cbb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:08.300166 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1265/2534] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:45:08 2022] DELETE /identity/v3/projects/71e0c71509f94f158063991e37617894 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:45:08.305075 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e740b-04be-4b54-86a0-8d3042ec6b19 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:08.305371 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e740b-04be-4b54-86a0-8d3042ec6b19 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:08.305650 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e740b-04be-4b54-86a0-8d3042ec6b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:08.323785 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-c89e740b-04be-4b54-86a0-8d3042ec6b19 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:45:08.324978 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-c89e740b-04be-4b54-86a0-8d3042ec6b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:08.397514 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1270/2535] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:45:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:45:08.405897 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb6cd1-2fdd-4d27-aa8b-bee5ccc6208a None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:08.406132 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb6cd1-2fdd-4d27-aa8b-bee5ccc6208a None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:08.406413 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb6cd1-2fdd-4d27-aa8b-bee5ccc6208a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:08.425562 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-98cb6cd1-2fdd-4d27-aa8b-bee5ccc6208a None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:45:08.426883 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-98cb6cd1-2fdd-4d27-aa8b-bee5ccc6208a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:08.434819 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1266/2536] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:45:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:45:08.439162 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e740b-04be-4b54-86a0-8d3042ec6b19 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:08.448768 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e740b-04be-4b54-86a0-8d3042ec6b19 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:08.454978 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0d2b7-1d92-4004-9483-3a6dd43fbc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:08.455780 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0d2b7-1d92-4004-9483-3a6dd43fbc92 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:08.456083 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0d2b7-1d92-4004-9483-3a6dd43fbc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:08.456511 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0d2b7-1d92-4004-9483-3a6dd43fbc92 admin admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:08.457550 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f0d2b7-1d92-4004-9483-3a6dd43fbc92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:45:08.457858 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f0d2b7-1d92-4004-9483-3a6dd43fbc92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:08.460072 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f0d2b7-1d92-4004-9483-3a6dd43fbc92 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:08.472779 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1271/2537] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:45:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:45:08.477290 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb6cd1-2fdd-4d27-aa8b-bee5ccc6208a None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:08.487330 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb6cd1-2fdd-4d27-aa8b-bee5ccc6208a None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:08.492104 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8713a-c2ec-462b-9e64-86e51e55e2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:08.493014 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8713a-c2ec-462b-9e64-86e51e55e2e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:08.493298 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8713a-c2ec-462b-9e64-86e51e55e2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:08.493690 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8713a-c2ec-462b-9e64-86e51e55e2e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:08.494424 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d8713a-c2ec-462b-9e64-86e51e55e2e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:08.496338 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d8713a-c2ec-462b-9e64-86e51e55e2e0 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:08.522198 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1267/2538] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:45:08 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:45:08.526123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0d2b7-1d92-4004-9483-3a6dd43fbc92 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:08.536705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0d2b7-1d92-4004-9483-3a6dd43fbc92 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:08.541325 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:08.595229 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd47efc-ee44-4612-82f1-9e716af6e9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:08.595900 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd47efc-ee44-4612-82f1-9e716af6e9bd admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:08.596186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd47efc-ee44-4612-82f1-9e716af6e9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:08.596468 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd47efc-ee44-4612-82f1-9e716af6e9bd admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:08.597166 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd47efc-ee44-4612-82f1-9e716af6e9bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:08.599109 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd47efc-ee44-4612-82f1-9e716af6e9bd admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:08.619121 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1272/2539] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:45:08 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:45:08.623364 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8713a-c2ec-462b-9e64-86e51e55e2e0 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:08.633680 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8713a-c2ec-462b-9e64-86e51e55e2e0 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:08.638990 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73822ac2-5123-4b2d-9cb3-c5af060ac365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:08.639656 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73822ac2-5123-4b2d-9cb3-c5af060ac365 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:08.639938 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73822ac2-5123-4b2d-9cb3-c5af060ac365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:08.640213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73822ac2-5123-4b2d-9cb3-c5af060ac365 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:08.640953 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73822ac2-5123-4b2d-9cb3-c5af060ac365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:08.642644 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73822ac2-5123-4b2d-9cb3-c5af060ac365 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:08.653111 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1268/2540] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:45:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:45:08.658338 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd47efc-ee44-4612-82f1-9e716af6e9bd admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:08.668165 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd47efc-ee44-4612-82f1-9e716af6e9bd admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:08.673293 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d299152e-6724-4a6a-9ab0-298919580450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:08.673946 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299152e-6724-4a6a-9ab0-298919580450 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:08.674268 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299152e-6724-4a6a-9ab0-298919580450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:08.674548 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299152e-6724-4a6a-9ab0-298919580450 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:08.675507 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d299152e-6724-4a6a-9ab0-298919580450 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:08.677096 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d299152e-6724-4a6a-9ab0-298919580450 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:08.687304 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1273/2541] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:45:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:45:08.691469 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73822ac2-5123-4b2d-9cb3-c5af060ac365 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:08.701925 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73822ac2-5123-4b2d-9cb3-c5af060ac365 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:08.706548 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e08f6d-ae45-4670-a141-04983bb9abb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:08.707423 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e08f6d-ae45-4670-a141-04983bb9abb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:08.707708 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e08f6d-ae45-4670-a141-04983bb9abb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:08.707983 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e08f6d-ae45-4670-a141-04983bb9abb6 admin admin] PATH_INFO: `/v3/projects/1d29e84f27fa4b709e09489042053c5b/users/51de7a18bcb44a779b633f8db30602ce/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:08.723510 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e08f6d-ae45-4670-a141-04983bb9abb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d29e84f27fa4b709e09489042053c5b, user_id=51de7a18bcb44a779b633f8db30602ce, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:08.726196 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e08f6d-ae45-4670-a141-04983bb9abb6 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:08.742876 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1269/2542] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:45:08 2022] PUT /identity/v3/projects/1d29e84f27fa4b709e09489042053c5b/users/51de7a18bcb44a779b633f8db30602ce/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:45:08.751986 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a227e-cbfa-48ed-8821-8d047d2f7b50 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:08.753119 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a227e-cbfa-48ed-8821-8d047d2f7b50 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:08.753419 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a227e-cbfa-48ed-8821-8d047d2f7b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:08.782616 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-323a227e-cbfa-48ed-8821-8d047d2f7b50 None None] MFA Rules not processed for user `51de7a18bcb44a779b633f8db30602ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:45:08.783995 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-323a227e-cbfa-48ed-8821-8d047d2f7b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:08.855788 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1274/2543] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:45:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:45:08.866765 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e08f6d-ae45-4670-a141-04983bb9abb6 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:08.868686 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d2e08f6d-ae45-4670-a141-04983bb9abb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:08.901089 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:08.948038 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e08f6d-ae45-4670-a141-04983bb9abb6 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:08.954972 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd1b8d-cc36-4559-97dc-8fc0498e9f1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:08.955627 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd1b8d-cc36-4559-97dc-8fc0498e9f1f service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:08.955914 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd1b8d-cc36-4559-97dc-8fc0498e9f1f service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:08.956191 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd1b8d-cc36-4559-97dc-8fc0498e9f1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:08.966409 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dd1b8d-cc36-4559-97dc-8fc0498e9f1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:08.968448 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dd1b8d-cc36-4559-97dc-8fc0498e9f1f service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:08.974441 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a227e-cbfa-48ed-8821-8d047d2f7b50 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:08.975673 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-323a227e-cbfa-48ed-8821-8d047d2f7b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:08.982270 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1270/2544] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:45:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:45:09.051045 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a227e-cbfa-48ed-8821-8d047d2f7b50 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:09.056298 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855e306a-7a20-45cc-a129-3924ff19bd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:09.057358 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e306a-7a20-45cc-a129-3924ff19bd55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:09.057651 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e306a-7a20-45cc-a129-3924ff19bd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:09.058041 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e306a-7a20-45cc-a129-3924ff19bd55 admin admin] PATH_INFO: `/v3/users/ca02102f415b41ae9263dc76a288ce41` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:09.059447 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855e306a-7a20-45cc-a129-3924ff19bd55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca02102f415b41ae9263dc76a288ce41)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:09.061482 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855e306a-7a20-45cc-a129-3924ff19bd55 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:09.086338 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:09.119049 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-855e306a-7a20-45cc-a129-3924ff19bd55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca02102f415b41ae9263dc76a288ce41'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:09.120969 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-855e306a-7a20-45cc-a129-3924ff19bd55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca02102f415b41ae9263dc76a288ce41'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:09.133451 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-855e306a-7a20-45cc-a129-3924ff19bd55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca02102f415b41ae9263dc76a288ce41'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:09.134045 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-855e306a-7a20-45cc-a129-3924ff19bd55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca02102f415b41ae9263dc76a288ce41'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:09.142369 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-855e306a-7a20-45cc-a129-3924ff19bd55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca02102f415b41ae9263dc76a288ce41'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:09.164793 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1275/2545] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:45:08 2022] DELETE /identity/v3/users/ca02102f415b41ae9263dc76a288ce41 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:45:09.169588 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1ce70-d8d2-450f-a898-962284cfb363 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:09.170146 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1ce70-d8d2-450f-a898-962284cfb363 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:09.170146 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1ce70-d8d2-450f-a898-962284cfb363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:09.200502 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-e4a1ce70-d8d2-450f-a898-962284cfb363 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:45:09.202582 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e4a1ce70-d8d2-450f-a898-962284cfb363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:09.282865 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1271/2546] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:45:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:45:09.293870 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855e306a-7a20-45cc-a129-3924ff19bd55 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:09.295996 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-855e306a-7a20-45cc-a129-3924ff19bd55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:09.386109 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855e306a-7a20-45cc-a129-3924ff19bd55 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:09.390384 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fadc09f-2396-4e45-a78a-3774b45100e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:09.391143 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fadc09f-2396-4e45-a78a-3774b45100e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:09.391555 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fadc09f-2396-4e45-a78a-3774b45100e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:09.392352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fadc09f-2396-4e45-a78a-3774b45100e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:09.408386 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fadc09f-2396-4e45-a78a-3774b45100e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:09.408386 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fadc09f-2396-4e45-a78a-3774b45100e9 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:09.416161 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1276/2547] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:45:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:45:09.704596 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1ce70-d8d2-450f-a898-962284cfb363 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:09.706015 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e4a1ce70-d8d2-450f-a898-962284cfb363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:09.720107 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1ce70-d8d2-450f-a898-962284cfb363 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:09.725040 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c03b63d-4ee5-4790-a1bb-3934c17e6560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:09.725937 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03b63d-4ee5-4790-a1bb-3934c17e6560 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:09.726270 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03b63d-4ee5-4790-a1bb-3934c17e6560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:09.726536 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03b63d-4ee5-4790-a1bb-3934c17e6560 admin admin] PATH_INFO: `/v3/projects/a2d3499fcfd34f3dac6ab8f3e31eea1e` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:09.728015 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c03b63d-4ee5-4790-a1bb-3934c17e6560 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2d3499fcfd34f3dac6ab8f3e31eea1e)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:09.729987 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c03b63d-4ee5-4790-a1bb-3934c17e6560 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:09.786278 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5c03b63d-4ee5-4790-a1bb-3934c17e6560 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2d3499fcfd34f3dac6ab8f3e31eea1e', 'request_id': 'req-5c03b63d-4ee5-4790-a1bb-3934c17e6560'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:09.795376 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5c03b63d-4ee5-4790-a1bb-3934c17e6560 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2d3499fcfd34f3dac6ab8f3e31eea1e', 'request_id': 'req-5c03b63d-4ee5-4790-a1bb-3934c17e6560'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:09.802702 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5c03b63d-4ee5-4790-a1bb-3934c17e6560 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2d3499fcfd34f3dac6ab8f3e31eea1e', 'request_id': 'req-5c03b63d-4ee5-4790-a1bb-3934c17e6560'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:09.814963 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5c03b63d-4ee5-4790-a1bb-3934c17e6560 admin admin] The token cache is being invalidate because project a2d3499fcfd34f3dac6ab8f3e31eea1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:45:09.815259 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5c03b63d-4ee5-4790-a1bb-3934c17e6560 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:09.815919 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5c03b63d-4ee5-4790-a1bb-3934c17e6560 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:09.817521 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1272/2548] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:45:09 2022] DELETE /identity/v3/projects/a2d3499fcfd34f3dac6ab8f3e31eea1e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:45:09.822865 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b2589-11b3-41a8-a19e-6253032a3761 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:09.823380 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b2589-11b3-41a8-a19e-6253032a3761 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:09.823481 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b2589-11b3-41a8-a19e-6253032a3761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:09.845980 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-eb5b2589-11b3-41a8-a19e-6253032a3761 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:45:09.847390 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-eb5b2589-11b3-41a8-a19e-6253032a3761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:09.911764 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1277/2549] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:45:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:45:09.919526 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156690e-9205-4bf6-9411-9f5ff7bdb9a4 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:09.919815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156690e-9205-4bf6-9411-9f5ff7bdb9a4 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:09.920098 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156690e-9205-4bf6-9411-9f5ff7bdb9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:09.939359 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-7156690e-9205-4bf6-9411-9f5ff7bdb9a4 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:45:09.940590 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7156690e-9205-4bf6-9411-9f5ff7bdb9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:09.948863 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1273/2550] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:45:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:45:09.953850 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5b2589-11b3-41a8-a19e-6253032a3761 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:09.964125 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5b2589-11b3-41a8-a19e-6253032a3761 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:09.969000 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44faec5-99be-4baa-ab55-cf707f38960a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:09.969695 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44faec5-99be-4baa-ab55-cf707f38960a admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:09.969985 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44faec5-99be-4baa-ab55-cf707f38960a admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:09.970298 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44faec5-99be-4baa-ab55-cf707f38960a admin admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:09.971070 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44faec5-99be-4baa-ab55-cf707f38960a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:45:09.971367 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44faec5-99be-4baa-ab55-cf707f38960a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:09.973017 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44faec5-99be-4baa-ab55-cf707f38960a admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:09.986784 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1278/2551] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:45:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:45:09.991377 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7156690e-9205-4bf6-9411-9f5ff7bdb9a4 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:10.002170 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7156690e-9205-4bf6-9411-9f5ff7bdb9a4 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:10.007152 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734b47a-dc34-474d-84ad-f66ace8cf2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:10.007898 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734b47a-dc34-474d-84ad-f66ace8cf2b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:10.008190 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734b47a-dc34-474d-84ad-f66ace8cf2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:10.008469 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734b47a-dc34-474d-84ad-f66ace8cf2b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:10.009203 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7734b47a-dc34-474d-84ad-f66ace8cf2b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:10.011238 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7734b47a-dc34-474d-84ad-f66ace8cf2b3 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:10.038919 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1274/2552] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:45:09 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:45:10.042995 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44faec5-99be-4baa-ab55-cf707f38960a admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:10.054271 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44faec5-99be-4baa-ab55-cf707f38960a admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:10.079554 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:10.114505 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f0c38-bdd7-4029-aa3a-a06c964835b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:10.115185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f0c38-bdd7-4029-aa3a-a06c964835b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:10.115469 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f0c38-bdd7-4029-aa3a-a06c964835b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:10.115747 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f0c38-bdd7-4029-aa3a-a06c964835b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:10.116445 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f0c38-bdd7-4029-aa3a-a06c964835b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:10.118664 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f0c38-bdd7-4029-aa3a-a06c964835b1 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:10.140505 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1279/2553] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:45:10 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:45:10.144678 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734b47a-dc34-474d-84ad-f66ace8cf2b3 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:10.155468 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734b47a-dc34-474d-84ad-f66ace8cf2b3 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:10.160083 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18b95f-7239-4cbe-82a0-3779327a2814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:10.160781 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18b95f-7239-4cbe-82a0-3779327a2814 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:10.161058 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18b95f-7239-4cbe-82a0-3779327a2814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:10.161331 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18b95f-7239-4cbe-82a0-3779327a2814 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:10.162057 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f18b95f-7239-4cbe-82a0-3779327a2814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:10.163705 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f18b95f-7239-4cbe-82a0-3779327a2814 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:10.165152 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:10.177342 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1275/2554] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:45:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:45:10.181522 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f0c38-bdd7-4029-aa3a-a06c964835b1 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:10.192099 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f0c38-bdd7-4029-aa3a-a06c964835b1 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:10.196652 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce098b-3093-418f-a8e4-f1c3e9ee3f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:10.197314 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce098b-3093-418f-a8e4-f1c3e9ee3f89 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:10.197616 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce098b-3093-418f-a8e4-f1c3e9ee3f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:10.197876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce098b-3093-418f-a8e4-f1c3e9ee3f89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:10.198642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ce098b-3093-418f-a8e4-f1c3e9ee3f89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:10.200299 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ce098b-3093-418f-a8e4-f1c3e9ee3f89 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:10.211237 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1280/2555] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:45:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:45:10.215563 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18b95f-7239-4cbe-82a0-3779327a2814 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:10.225569 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18b95f-7239-4cbe-82a0-3779327a2814 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:10.230218 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d8fc0-337c-4a98-bafd-3712f8f88516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:10.231097 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d8fc0-337c-4a98-bafd-3712f8f88516 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:10.231376 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d8fc0-337c-4a98-bafd-3712f8f88516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:10.231645 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d8fc0-337c-4a98-bafd-3712f8f88516 admin admin] PATH_INFO: `/v3/projects/98d1f507630f433a88495bac1b3235a4/users/8e8153a1c4e54cfcbecf8a9bac4ade86/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:10.247379 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22d8fc0-337c-4a98-bafd-3712f8f88516 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98d1f507630f433a88495bac1b3235a4, user_id=8e8153a1c4e54cfcbecf8a9bac4ade86, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:10.250622 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22d8fc0-337c-4a98-bafd-3712f8f88516 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:10.265340 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1276/2556] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:45:10 2022] PUT /identity/v3/projects/98d1f507630f433a88495bac1b3235a4/users/8e8153a1c4e54cfcbecf8a9bac4ade86/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:45:10.276405 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a36563-ad98-473e-8846-fa7cb1aca908 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:10.276699 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a36563-ad98-473e-8846-fa7cb1aca908 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:10.276979 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a36563-ad98-473e-8846-fa7cb1aca908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:10.306843 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-81a36563-ad98-473e-8846-fa7cb1aca908 None None] MFA Rules not processed for user `8e8153a1c4e54cfcbecf8a9bac4ade86`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:45:10.308079 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-81a36563-ad98-473e-8846-fa7cb1aca908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:10.383776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1281/2557] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:45:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:45:10.388548 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d8fc0-337c-4a98-bafd-3712f8f88516 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:10.478657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d8fc0-337c-4a98-bafd-3712f8f88516 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:10.483677 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0b27d-9699-4139-a91d-90c9e6a3be83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:10.485286 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0b27d-9699-4139-a91d-90c9e6a3be83 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:10.485286 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0b27d-9699-4139-a91d-90c9e6a3be83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:10.485566 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0b27d-9699-4139-a91d-90c9e6a3be83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:10.486586 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f0b27d-9699-4139-a91d-90c9e6a3be83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:10.489087 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f0b27d-9699-4139-a91d-90c9e6a3be83 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:10.525963 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1277/2558] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:45:10 2022] POST /identity/v3/projects => generated 385 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:45:10.530561 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a36563-ad98-473e-8846-fa7cb1aca908 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:10.542875 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a36563-ad98-473e-8846-fa7cb1aca908 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:10.623445 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5e7150-f6a4-4928-a4de-22c04ae7e906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:10.624114 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e7150-f6a4-4928-a4de-22c04ae7e906 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:10.624398 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e7150-f6a4-4928-a4de-22c04ae7e906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:10.624746 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e7150-f6a4-4928-a4de-22c04ae7e906 admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:10.625793 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5e7150-f6a4-4928-a4de-22c04ae7e906 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:10.628382 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5e7150-f6a4-4928-a4de-22c04ae7e906 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:10.652061 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1282/2559] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:45:10 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:45:10.657025 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0b27d-9699-4139-a91d-90c9e6a3be83 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:10.670321 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0b27d-9699-4139-a91d-90c9e6a3be83 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:10.676690 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024078c-bce6-40ed-9a6b-b3e100a700bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:10.677774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024078c-bce6-40ed-9a6b-b3e100a700bb admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:10.678455 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024078c-bce6-40ed-9a6b-b3e100a700bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:10.678455 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024078c-bce6-40ed-9a6b-b3e100a700bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:10.679450 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024078c-bce6-40ed-9a6b-b3e100a700bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:10.680970 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024078c-bce6-40ed-9a6b-b3e100a700bb admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:10.693626 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1278/2560] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:45:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:45:10.697709 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5e7150-f6a4-4928-a4de-22c04ae7e906 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:10.708544 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5e7150-f6a4-4928-a4de-22c04ae7e906 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:10.713505 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea043f7-3a02-409a-954d-10e36a5a87e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:10.714178 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea043f7-3a02-409a-954d-10e36a5a87e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:10.714600 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea043f7-3a02-409a-954d-10e36a5a87e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:10.714853 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea043f7-3a02-409a-954d-10e36a5a87e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:10.715656 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea043f7-3a02-409a-954d-10e36a5a87e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:10.717939 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea043f7-3a02-409a-954d-10e36a5a87e7 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:10.729191 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1283/2561] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:45:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:45:10.734666 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024078c-bce6-40ed-9a6b-b3e100a700bb admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:10.745635 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024078c-bce6-40ed-9a6b-b3e100a700bb admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:10.751269 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8840e09-d0b2-40f1-ba29-335637c17a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:10.752174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8840e09-d0b2-40f1-ba29-335637c17a46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:10.752456 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8840e09-d0b2-40f1-ba29-335637c17a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:10.752864 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8840e09-d0b2-40f1-ba29-335637c17a46 admin admin] PATH_INFO: `/v3/projects/ec7426bd444d411a916e2b836c9e60e6/users/685b927bda6c4c319ec7a30773f27b1a/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:10.770662 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8840e09-d0b2-40f1-ba29-335637c17a46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec7426bd444d411a916e2b836c9e60e6, user_id=685b927bda6c4c319ec7a30773f27b1a, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:10.773739 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8840e09-d0b2-40f1-ba29-335637c17a46 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:10.789141 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1279/2562] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:45:10 2022] PUT /identity/v3/projects/ec7426bd444d411a916e2b836c9e60e6/users/685b927bda6c4c319ec7a30773f27b1a/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:45:10.801171 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491fa40f-995e-4081-a05c-17625b14eaee None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:10.801471 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491fa40f-995e-4081-a05c-17625b14eaee None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:10.801747 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491fa40f-995e-4081-a05c-17625b14eaee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:10.832037 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:10.835564 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-491fa40f-995e-4081-a05c-17625b14eaee None None] MFA Rules not processed for user `685b927bda6c4c319ec7a30773f27b1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:45:10.837560 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-491fa40f-995e-4081-a05c-17625b14eaee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:10.909680 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1284/2563] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:45:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:45:10.914905 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8840e09-d0b2-40f1-ba29-335637c17a46 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:10.984492 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8840e09-d0b2-40f1-ba29-335637c17a46 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:10.990479 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02759d-de84-4fbd-9195-b4d7d9db73ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:10.991333 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02759d-de84-4fbd-9195-b4d7d9db73ea admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:10.991750 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02759d-de84-4fbd-9195-b4d7d9db73ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:10.992148 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02759d-de84-4fbd-9195-b4d7d9db73ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:10.993232 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02759d-de84-4fbd-9195-b4d7d9db73ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:10.995193 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02759d-de84-4fbd-9195-b4d7d9db73ea admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:11.023007 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1280/2564] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:45:10 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:45:11.027328 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491fa40f-995e-4081-a05c-17625b14eaee None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:11.038792 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491fa40f-995e-4081-a05c-17625b14eaee None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:11.097459 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba6ad4-57aa-447f-ad69-8a73c3d0e444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:11.098170 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba6ad4-57aa-447f-ad69-8a73c3d0e444 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:11.098458 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba6ad4-57aa-447f-ad69-8a73c3d0e444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:11.098736 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba6ad4-57aa-447f-ad69-8a73c3d0e444 admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:11.099433 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ba6ad4-57aa-447f-ad69-8a73c3d0e444 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:11.102067 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ba6ad4-57aa-447f-ad69-8a73c3d0e444 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:11.126854 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1285/2565] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:45:11 2022] POST /identity/v3/users => generated 298 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:45:11.130920 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02759d-de84-4fbd-9195-b4d7d9db73ea admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:11.143228 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02759d-de84-4fbd-9195-b4d7d9db73ea admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:11.148437 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedce90c-c9e4-4ad4-af83-a93b37f542b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:11.149134 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedce90c-c9e4-4ad4-af83-a93b37f542b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:11.149427 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedce90c-c9e4-4ad4-af83-a93b37f542b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:11.149707 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedce90c-c9e4-4ad4-af83-a93b37f542b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:11.150523 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedce90c-c9e4-4ad4-af83-a93b37f542b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:11.152234 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedce90c-c9e4-4ad4-af83-a93b37f542b3 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:11.154450 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:11.165890 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1281/2566] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:45:11.170812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba6ad4-57aa-447f-ad69-8a73c3d0e444 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:11.181227 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba6ad4-57aa-447f-ad69-8a73c3d0e444 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:11.185952 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6ac94-c716-4525-aeae-ed81b4f11a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:11.186896 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6ac94-c716-4525-aeae-ed81b4f11a1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:11.187181 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6ac94-c716-4525-aeae-ed81b4f11a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:11.187454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6ac94-c716-4525-aeae-ed81b4f11a1e admin admin] PATH_INFO: `/v3/projects/0ddae7b20db14afba00a91dffdeb761f/users/12c056ee9d0b4d58bfb7af1bb147a3c9/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:11.203925 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a6ac94-c716-4525-aeae-ed81b4f11a1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ddae7b20db14afba00a91dffdeb761f, user_id=12c056ee9d0b4d58bfb7af1bb147a3c9, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:11.206693 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a6ac94-c716-4525-aeae-ed81b4f11a1e admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:11.221261 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1286/2567] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:45:11 2022] PUT /identity/v3/projects/0ddae7b20db14afba00a91dffdeb761f/users/12c056ee9d0b4d58bfb7af1bb147a3c9/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:45:11.225343 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedce90c-c9e4-4ad4-af83-a93b37f542b3 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:11.236840 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedce90c-c9e4-4ad4-af83-a93b37f542b3 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:11.298465 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0458874-28a1-47d3-8f6a-d5778cc1ed0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:11.299128 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0458874-28a1-47d3-8f6a-d5778cc1ed0f admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:11.299414 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0458874-28a1-47d3-8f6a-d5778cc1ed0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:11.299693 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0458874-28a1-47d3-8f6a-d5778cc1ed0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:11.300420 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0458874-28a1-47d3-8f6a-d5778cc1ed0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:11.302016 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0458874-28a1-47d3-8f6a-d5778cc1ed0f admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:11.311684 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1282/2568] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:45:11.315908 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6ac94-c716-4525-aeae-ed81b4f11a1e admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:11.326227 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6ac94-c716-4525-aeae-ed81b4f11a1e admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:11.330871 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d5c331-ff34-41ae-96e4-28f9426481af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:11.331749 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5c331-ff34-41ae-96e4-28f9426481af admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:11.332030 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5c331-ff34-41ae-96e4-28f9426481af admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:11.332306 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5c331-ff34-41ae-96e4-28f9426481af admin admin] PATH_INFO: `/v3/projects/0ddae7b20db14afba00a91dffdeb761f/users/12c056ee9d0b4d58bfb7af1bb147a3c9/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:11.334287 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d5c331-ff34-41ae-96e4-28f9426481af admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ddae7b20db14afba00a91dffdeb761f, user_id=12c056ee9d0b4d58bfb7af1bb147a3c9, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:11.336901 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d5c331-ff34-41ae-96e4-28f9426481af admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:11.348316 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1287/2569] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:45:11 2022] PUT /identity/v3/projects/0ddae7b20db14afba00a91dffdeb761f/users/12c056ee9d0b4d58bfb7af1bb147a3c9/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:45:11.360139 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8211ac4d-694c-4ec3-82ef-7f6ba2531e16 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:11.360438 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8211ac4d-694c-4ec3-82ef-7f6ba2531e16 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:11.360721 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8211ac4d-694c-4ec3-82ef-7f6ba2531e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:11.388224 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-8211ac4d-694c-4ec3-82ef-7f6ba2531e16 None None] MFA Rules not processed for user `12c056ee9d0b4d58bfb7af1bb147a3c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:45:11.389444 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8211ac4d-694c-4ec3-82ef-7f6ba2531e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:11.458249 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1283/2570] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:45:11 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:45:11.469571 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d5c331-ff34-41ae-96e4-28f9426481af admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:11.471303 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-96d5c331-ff34-41ae-96e4-28f9426481af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:11.529672 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d5c331-ff34-41ae-96e4-28f9426481af admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:11.534012 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff86ed-a4e8-4acc-abf8-c9a9995701a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:11.534688 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff86ed-a4e8-4acc-abf8-c9a9995701a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:11.534971 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff86ed-a4e8-4acc-abf8-c9a9995701a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:11.535247 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff86ed-a4e8-4acc-abf8-c9a9995701a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:11.543700 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:11.599745 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cff86ed-a4e8-4acc-abf8-c9a9995701a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:11.601177 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cff86ed-a4e8-4acc-abf8-c9a9995701a7 service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:11.610825 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1288/2571] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:45:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:45:11.822166 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8211ac4d-694c-4ec3-82ef-7f6ba2531e16 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:11.822166 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8211ac4d-694c-4ec3-82ef-7f6ba2531e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:11.928820 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8211ac4d-694c-4ec3-82ef-7f6ba2531e16 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:11.933600 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a235fd36-f314-4a3e-a955-5a8f891b9b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:11.934521 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235fd36-f314-4a3e-a955-5a8f891b9b7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:11.934845 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235fd36-f314-4a3e-a955-5a8f891b9b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:11.935125 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235fd36-f314-4a3e-a955-5a8f891b9b7f admin admin] PATH_INFO: `/v3/users/51de7a18bcb44a779b633f8db30602ce` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:11.936503 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a235fd36-f314-4a3e-a955-5a8f891b9b7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=51de7a18bcb44a779b633f8db30602ce)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:11.938492 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a235fd36-f314-4a3e-a955-5a8f891b9b7f admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:11.962015 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:12.001226 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-a235fd36-f314-4a3e-a955-5a8f891b9b7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51de7a18bcb44a779b633f8db30602ce'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:12.001996 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-a235fd36-f314-4a3e-a955-5a8f891b9b7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51de7a18bcb44a779b633f8db30602ce'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:12.002722 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-a235fd36-f314-4a3e-a955-5a8f891b9b7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51de7a18bcb44a779b633f8db30602ce'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:12.020403 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-a235fd36-f314-4a3e-a955-5a8f891b9b7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51de7a18bcb44a779b633f8db30602ce'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:12.029385 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-a235fd36-f314-4a3e-a955-5a8f891b9b7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51de7a18bcb44a779b633f8db30602ce'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:12.045403 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1284/2572] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:45:11 2022] DELETE /identity/v3/users/51de7a18bcb44a779b633f8db30602ce => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:45:12.049882 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d8fbb-cad8-41e1-bae0-932005eff450 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:12.050403 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d8fbb-cad8-41e1-bae0-932005eff450 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:12.050888 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d8fbb-cad8-41e1-bae0-932005eff450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:12.069795 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-530d8fbb-cad8-41e1-bae0-932005eff450 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:45:12.071068 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-530d8fbb-cad8-41e1-bae0-932005eff450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:12.131334 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1289/2573] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:45:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:45:12.141886 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a235fd36-f314-4a3e-a955-5a8f891b9b7f admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:12.143496 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a235fd36-f314-4a3e-a955-5a8f891b9b7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:12.205222 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a235fd36-f314-4a3e-a955-5a8f891b9b7f admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:12.209735 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d808b96-ff26-44e1-b2c3-71a074194e52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:12.210417 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d808b96-ff26-44e1-b2c3-71a074194e52 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:12.210701 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d808b96-ff26-44e1-b2c3-71a074194e52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:12.210977 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d808b96-ff26-44e1-b2c3-71a074194e52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:12.221859 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d808b96-ff26-44e1-b2c3-71a074194e52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:12.223438 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d808b96-ff26-44e1-b2c3-71a074194e52 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:12.234346 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1285/2574] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:45:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:45:12.547925 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d8fbb-cad8-41e1-bae0-932005eff450 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:12.549594 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-530d8fbb-cad8-41e1-bae0-932005eff450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:12.573691 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d8fbb-cad8-41e1-bae0-932005eff450 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:12.579759 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421e7bd-c2b4-40c6-850f-61c7cf91c12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:12.580648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421e7bd-c2b4-40c6-850f-61c7cf91c12a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:12.580938 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421e7bd-c2b4-40c6-850f-61c7cf91c12a admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:12.581219 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421e7bd-c2b4-40c6-850f-61c7cf91c12a admin admin] PATH_INFO: `/v3/projects/1d29e84f27fa4b709e09489042053c5b` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:12.582496 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d421e7bd-c2b4-40c6-850f-61c7cf91c12a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d29e84f27fa4b709e09489042053c5b)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:12.584418 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d421e7bd-c2b4-40c6-850f-61c7cf91c12a admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:12.635109 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-d421e7bd-c2b4-40c6-850f-61c7cf91c12a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d29e84f27fa4b709e09489042053c5b', 'request_id': 'req-d421e7bd-c2b4-40c6-850f-61c7cf91c12a'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:12.643784 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-d421e7bd-c2b4-40c6-850f-61c7cf91c12a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d29e84f27fa4b709e09489042053c5b', 'request_id': 'req-d421e7bd-c2b4-40c6-850f-61c7cf91c12a'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:12.650924 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-d421e7bd-c2b4-40c6-850f-61c7cf91c12a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d29e84f27fa4b709e09489042053c5b', 'request_id': 'req-d421e7bd-c2b4-40c6-850f-61c7cf91c12a'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:12.662464 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-d421e7bd-c2b4-40c6-850f-61c7cf91c12a admin admin] The token cache is being invalidate because project 1d29e84f27fa4b709e09489042053c5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:45:12.662753 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-d421e7bd-c2b4-40c6-850f-61c7cf91c12a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:12.663413 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-d421e7bd-c2b4-40c6-850f-61c7cf91c12a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:12.665019 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1290/2575] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:45:12 2022] DELETE /identity/v3/projects/1d29e84f27fa4b709e09489042053c5b => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:45:12.672758 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608207f-195a-4c81-86b4-b50bd53f1e2c None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:12.673044 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608207f-195a-4c81-86b4-b50bd53f1e2c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:12.673321 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608207f-195a-4c81-86b4-b50bd53f1e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:12.691912 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-7608207f-195a-4c81-86b4-b50bd53f1e2c None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:45:12.693141 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7608207f-195a-4c81-86b4-b50bd53f1e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:12.755544 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1286/2576] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:45:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:45:12.763967 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5092a1-d626-4a64-bbfd-c8849611540d None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:12.764256 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5092a1-d626-4a64-bbfd-c8849611540d None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:12.764537 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5092a1-d626-4a64-bbfd-c8849611540d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:12.784065 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-3f5092a1-d626-4a64-bbfd-c8849611540d None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:45:12.785284 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-3f5092a1-d626-4a64-bbfd-c8849611540d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:12.793384 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1291/2577] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:45:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:45:12.798355 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608207f-195a-4c81-86b4-b50bd53f1e2c None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:12.810549 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608207f-195a-4c81-86b4-b50bd53f1e2c None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:12.815421 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff1c7a-011b-4dcc-9805-a03fb80b2b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:12.816107 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff1c7a-011b-4dcc-9805-a03fb80b2b68 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:12.816397 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff1c7a-011b-4dcc-9805-a03fb80b2b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:12.816680 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff1c7a-011b-4dcc-9805-a03fb80b2b68 admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:12.817431 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ff1c7a-011b-4dcc-9805-a03fb80b2b68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:45:12.817743 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ff1c7a-011b-4dcc-9805-a03fb80b2b68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:12.819409 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ff1c7a-011b-4dcc-9805-a03fb80b2b68 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:12.833862 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1287/2578] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:45:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:45:12.838360 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5092a1-d626-4a64-bbfd-c8849611540d None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:12.848652 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5092a1-d626-4a64-bbfd-c8849611540d None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:12.853372 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41134478-3167-4ff5-8f86-c4ddd2f6a5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:12.854035 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41134478-3167-4ff5-8f86-c4ddd2f6a5a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:12.854356 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41134478-3167-4ff5-8f86-c4ddd2f6a5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:12.854639 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41134478-3167-4ff5-8f86-c4ddd2f6a5a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:12.855355 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41134478-3167-4ff5-8f86-c4ddd2f6a5a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:12.857845 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41134478-3167-4ff5-8f86-c4ddd2f6a5a2 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:12.887339 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1292/2579] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:45:12 2022] POST /identity/v3/projects => generated 399 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:45:12.892033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff1c7a-011b-4dcc-9805-a03fb80b2b68 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:12.902902 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff1c7a-011b-4dcc-9805-a03fb80b2b68 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:12.936016 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:12.996072 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e0246a-b899-499e-b400-338213659254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:12.998395 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e0246a-b899-499e-b400-338213659254 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:12.998683 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e0246a-b899-499e-b400-338213659254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:12.998961 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e0246a-b899-499e-b400-338213659254 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:12.999684 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e0246a-b899-499e-b400-338213659254 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:13.001741 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e0246a-b899-499e-b400-338213659254 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:13.034136 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1288/2580] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:45:12 2022] POST /identity/v3/users => generated 300 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:45:13.037958 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41134478-3167-4ff5-8f86-c4ddd2f6a5a2 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:13.046933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:13.049950 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41134478-3167-4ff5-8f86-c4ddd2f6a5a2 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:13.057763 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae07750-bd3a-4545-9514-951f8771632e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:13.059271 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae07750-bd3a-4545-9514-951f8771632e admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:13.059902 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae07750-bd3a-4545-9514-951f8771632e admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:13.060373 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae07750-bd3a-4545-9514-951f8771632e admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:13.061410 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae07750-bd3a-4545-9514-951f8771632e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:13.063265 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae07750-bd3a-4545-9514-951f8771632e admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:13.077899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1293/2581] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:45:13.082389 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e0246a-b899-499e-b400-338213659254 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:13.094956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e0246a-b899-499e-b400-338213659254 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:13.101677 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a3e838-a735-4068-84c8-cd8a7b608178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:13.102489 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3e838-a735-4068-84c8-cd8a7b608178 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:13.102746 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3e838-a735-4068-84c8-cd8a7b608178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:13.103077 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3e838-a735-4068-84c8-cd8a7b608178 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:13.103838 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a3e838-a735-4068-84c8-cd8a7b608178 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:13.105468 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a3e838-a735-4068-84c8-cd8a7b608178 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:13.117626 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1289/2582] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:45:13.122056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae07750-bd3a-4545-9514-951f8771632e admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:13.134419 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae07750-bd3a-4545-9514-951f8771632e admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:13.140098 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55f3e4-b7ae-42d7-aeee-94cf7685de35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:13.141127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55f3e4-b7ae-42d7-aeee-94cf7685de35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:13.141460 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55f3e4-b7ae-42d7-aeee-94cf7685de35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:13.141740 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55f3e4-b7ae-42d7-aeee-94cf7685de35 admin admin] PATH_INFO: `/v3/projects/38b06ef0c29244659a14cf85f67345dc/users/9e012c8615ac4eb4b07352e1e4bbef5d/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:13.159730 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a55f3e4-b7ae-42d7-aeee-94cf7685de35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38b06ef0c29244659a14cf85f67345dc, user_id=9e012c8615ac4eb4b07352e1e4bbef5d, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:13.163151 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a55f3e4-b7ae-42d7-aeee-94cf7685de35 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:13.183532 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1294/2583] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:45:13 2022] PUT /identity/v3/projects/38b06ef0c29244659a14cf85f67345dc/users/9e012c8615ac4eb4b07352e1e4bbef5d/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:45:13.196207 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bf7d9-76fc-4ade-8b97-c517db2e4860 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:13.196505 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bf7d9-76fc-4ade-8b97-c517db2e4860 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:13.196792 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bf7d9-76fc-4ade-8b97-c517db2e4860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:13.198422 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55f3e4-b7ae-42d7-aeee-94cf7685de35 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:13.200537 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-5a55f3e4-b7ae-42d7-aeee-94cf7685de35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:13.236792 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-561bf7d9-76fc-4ade-8b97-c517db2e4860 None None] MFA Rules not processed for user `9e012c8615ac4eb4b07352e1e4bbef5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:45:13.238163 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-561bf7d9-76fc-4ade-8b97-c517db2e4860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:13.265217 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55f3e4-b7ae-42d7-aeee-94cf7685de35 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:13.269325 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec5561-2d4e-4b91-acb0-76ebf83bd058 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:13.270017 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec5561-2d4e-4b91-acb0-76ebf83bd058 service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:13.270310 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec5561-2d4e-4b91-acb0-76ebf83bd058 service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:13.270620 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec5561-2d4e-4b91-acb0-76ebf83bd058 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:13.272785 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-8aec5561-2d4e-4b91-acb0-76ebf83bd058 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:13.316200 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1290/2584] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:45:13 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:45:13.320892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561bf7d9-76fc-4ade-8b97-c517db2e4860 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:13.346166 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec5561-2d4e-4b91-acb0-76ebf83bd058 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:13.347831 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec5561-2d4e-4b91-acb0-76ebf83bd058 service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:13.368369 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1295/2585] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:45:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:45:13.417655 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561bf7d9-76fc-4ade-8b97-c517db2e4860 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:13.422828 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9048265-c3ee-497b-8e55-8525f0f015dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:13.423508 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9048265-c3ee-497b-8e55-8525f0f015dd admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:13.423790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9048265-c3ee-497b-8e55-8525f0f015dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:13.424063 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9048265-c3ee-497b-8e55-8525f0f015dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:13.424774 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9048265-c3ee-497b-8e55-8525f0f015dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:13.426759 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9048265-c3ee-497b-8e55-8525f0f015dd admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:13.462014 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1291/2586] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:45:13 2022] POST /identity/v3/projects => generated 405 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:45:13.466143 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec5561-2d4e-4b91-acb0-76ebf83bd058 service cinder] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:13.480727 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec5561-2d4e-4b91-acb0-76ebf83bd058 service cinder] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:13.555939 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b434d70-8895-4a46-a50f-141b2dc3a153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:13.556996 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b434d70-8895-4a46-a50f-141b2dc3a153 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:13.557571 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b434d70-8895-4a46-a50f-141b2dc3a153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:13.558011 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b434d70-8895-4a46-a50f-141b2dc3a153 admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:13.558835 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b434d70-8895-4a46-a50f-141b2dc3a153 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:13.563383 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b434d70-8895-4a46-a50f-141b2dc3a153 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:13.590818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1296/2587] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:45:13 2022] POST /identity/v3/users => generated 309 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:45:13.595321 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9048265-c3ee-497b-8e55-8525f0f015dd admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:13.608956 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9048265-c3ee-497b-8e55-8525f0f015dd admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:13.615340 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5d3c5-4214-4a00-8f5a-eeb4fefef918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:13.616382 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5d3c5-4214-4a00-8f5a-eeb4fefef918 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:13.617515 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5d3c5-4214-4a00-8f5a-eeb4fefef918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:13.617515 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5d3c5-4214-4a00-8f5a-eeb4fefef918 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:13.618519 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b5d3c5-4214-4a00-8f5a-eeb4fefef918 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:13.621916 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b5d3c5-4214-4a00-8f5a-eeb4fefef918 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:13.636344 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1292/2588] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:45:13.641506 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b434d70-8895-4a46-a50f-141b2dc3a153 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:13.654585 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b434d70-8895-4a46-a50f-141b2dc3a153 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:13.660144 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52499e-10de-4adb-83e3-4edde40ab293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:13.661211 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52499e-10de-4adb-83e3-4edde40ab293 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:13.661592 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52499e-10de-4adb-83e3-4edde40ab293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:13.661943 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52499e-10de-4adb-83e3-4edde40ab293 admin admin] PATH_INFO: `/v3/projects/04e6fbe2dc2b4608a62f4ebe4db882a2/users/0895090e443a41afbe0a41be2076ffe2/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:13.674856 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:13.679397 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d52499e-10de-4adb-83e3-4edde40ab293 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04e6fbe2dc2b4608a62f4ebe4db882a2, user_id=0895090e443a41afbe0a41be2076ffe2, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:13.683037 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d52499e-10de-4adb-83e3-4edde40ab293 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:13.697204 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1297/2589] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:45:13 2022] PUT /identity/v3/projects/04e6fbe2dc2b4608a62f4ebe4db882a2/users/0895090e443a41afbe0a41be2076ffe2/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:45:13.702123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5d3c5-4214-4a00-8f5a-eeb4fefef918 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:13.712386 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5d3c5-4214-4a00-8f5a-eeb4fefef918 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:13.718552 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:13.779666 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa0e34f-0f3a-4a60-b74f-facf6b87300c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:13.780445 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0e34f-0f3a-4a60-b74f-facf6b87300c admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:13.780755 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0e34f-0f3a-4a60-b74f-facf6b87300c admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:13.781043 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0e34f-0f3a-4a60-b74f-facf6b87300c admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:13.781804 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa0e34f-0f3a-4a60-b74f-facf6b87300c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:13.783556 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa0e34f-0f3a-4a60-b74f-facf6b87300c admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:13.799512 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1293/2590] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:45:13.801032 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52499e-10de-4adb-83e3-4edde40ab293 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:13.812671 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52499e-10de-4adb-83e3-4edde40ab293 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:13.817897 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82da30a-5bb8-4e00-854e-98d85e998b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:13.818819 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82da30a-5bb8-4e00-854e-98d85e998b6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:13.819108 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82da30a-5bb8-4e00-854e-98d85e998b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:13.819386 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82da30a-5bb8-4e00-854e-98d85e998b6c admin admin] PATH_INFO: `/v3/projects/04e6fbe2dc2b4608a62f4ebe4db882a2/users/0895090e443a41afbe0a41be2076ffe2/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:13.821386 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82da30a-5bb8-4e00-854e-98d85e998b6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=04e6fbe2dc2b4608a62f4ebe4db882a2, user_id=0895090e443a41afbe0a41be2076ffe2, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:13.824236 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82da30a-5bb8-4e00-854e-98d85e998b6c admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:13.837735 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1298/2591] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:45:13 2022] PUT /identity/v3/projects/04e6fbe2dc2b4608a62f4ebe4db882a2/users/0895090e443a41afbe0a41be2076ffe2/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:45:13.848574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2199db-f60b-4c17-b2fa-06247cdc8750 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:13.848870 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2199db-f60b-4c17-b2fa-06247cdc8750 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:13.849153 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2199db-f60b-4c17-b2fa-06247cdc8750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:13.878553 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-2e2199db-f60b-4c17-b2fa-06247cdc8750 None None] MFA Rules not processed for user `0895090e443a41afbe0a41be2076ffe2`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:45:13.879808 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-2e2199db-f60b-4c17-b2fa-06247cdc8750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:13.955778 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1294/2592] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:45:13 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:45:13.968692 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82da30a-5bb8-4e00-854e-98d85e998b6c admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:13.977645 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2199db-f60b-4c17-b2fa-06247cdc8750 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:13.979038 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-2e2199db-f60b-4c17-b2fa-06247cdc8750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:14.027182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82da30a-5bb8-4e00-854e-98d85e998b6c admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:14.031414 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779e7f9-bb3d-4191-8629-c8ad51459fd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:14.032151 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779e7f9-bb3d-4191-8629-c8ad51459fd4 service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:14.032436 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779e7f9-bb3d-4191-8629-c8ad51459fd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:14.032713 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779e7f9-bb3d-4191-8629-c8ad51459fd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:14.042954 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c779e7f9-bb3d-4191-8629-c8ad51459fd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:14.043825 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2199db-f60b-4c17-b2fa-06247cdc8750 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:14.044445 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c779e7f9-bb3d-4191-8629-c8ad51459fd4 service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:14.047719 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac420b-c502-4f0b-a2c4-7e7197f23649 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:14.048360 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac420b-c502-4f0b-a2c4-7e7197f23649 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:14.048650 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac420b-c502-4f0b-a2c4-7e7197f23649 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:14.048921 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac420b-c502-4f0b-a2c4-7e7197f23649 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:14.050775 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-31ac420b-c502-4f0b-a2c4-7e7197f23649 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:14.054657 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1299/2593] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:45:13 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:45:14.101178 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:14.123897 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ac420b-c502-4f0b-a2c4-7e7197f23649 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:14.125345 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ac420b-c502-4f0b-a2c4-7e7197f23649 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:14.135169 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1295/2594] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:45:13 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:45:14.189334 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779e7f9-bb3d-4191-8629-c8ad51459fd4 service cinder] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:14.266282 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779e7f9-bb3d-4191-8629-c8ad51459fd4 service cinder] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:14.270864 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a2abe4-41c8-409f-afc2-15efd958a36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:14.271732 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a2abe4-41c8-409f-afc2-15efd958a36e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:14.272009 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a2abe4-41c8-409f-afc2-15efd958a36e admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:14.272281 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a2abe4-41c8-409f-afc2-15efd958a36e admin admin] PATH_INFO: `/v3/users/9e012c8615ac4eb4b07352e1e4bbef5d` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:14.273478 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a2abe4-41c8-409f-afc2-15efd958a36e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e012c8615ac4eb4b07352e1e4bbef5d)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:14.275404 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a2abe4-41c8-409f-afc2-15efd958a36e admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:14.322595 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-e9a2abe4-41c8-409f-afc2-15efd958a36e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e012c8615ac4eb4b07352e1e4bbef5d'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:14.323365 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-e9a2abe4-41c8-409f-afc2-15efd958a36e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e012c8615ac4eb4b07352e1e4bbef5d'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:14.331656 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-e9a2abe4-41c8-409f-afc2-15efd958a36e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e012c8615ac4eb4b07352e1e4bbef5d'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:14.332349 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-e9a2abe4-41c8-409f-afc2-15efd958a36e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e012c8615ac4eb4b07352e1e4bbef5d'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:14.340712 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-e9a2abe4-41c8-409f-afc2-15efd958a36e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e012c8615ac4eb4b07352e1e4bbef5d'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:14.355047 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:14.362137 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1300/2595] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:45:14 2022] DELETE /identity/v3/users/9e012c8615ac4eb4b07352e1e4bbef5d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:45:14.367563 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b42f6-28de-47f5-b841-1834cd05e398 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:14.368112 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b42f6-28de-47f5-b841-1834cd05e398 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:14.368375 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b42f6-28de-47f5-b841-1834cd05e398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:14.385504 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-789b42f6-28de-47f5-b841-1834cd05e398 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:45:14.386806 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-789b42f6-28de-47f5-b841-1834cd05e398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:14.448887 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1296/2596] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:45:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:45:14.458478 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a2abe4-41c8-409f-afc2-15efd958a36e admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:14.459914 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-e9a2abe4-41c8-409f-afc2-15efd958a36e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:14.519110 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a2abe4-41c8-409f-afc2-15efd958a36e admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:14.522736 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5283c11-058b-424b-a78b-12a0bddf7747 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:14.523382 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5283c11-058b-424b-a78b-12a0bddf7747 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:14.523665 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5283c11-058b-424b-a78b-12a0bddf7747 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:14.523941 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5283c11-058b-424b-a78b-12a0bddf7747 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:14.533009 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5283c11-058b-424b-a78b-12a0bddf7747 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:14.534469 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5283c11-058b-424b-a78b-12a0bddf7747 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:14.543770 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1301/2597] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:45:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:45:14.825747 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b42f6-28de-47f5-b841-1834cd05e398 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:14.827409 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-789b42f6-28de-47f5-b841-1834cd05e398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:14.846556 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b42f6-28de-47f5-b841-1834cd05e398 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:14.858182 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d382f6-d75b-4654-82d0-c6f172abbd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:14.859061 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d382f6-d75b-4654-82d0-c6f172abbd7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:14.859342 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d382f6-d75b-4654-82d0-c6f172abbd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:14.859620 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d382f6-d75b-4654-82d0-c6f172abbd7a admin admin] PATH_INFO: `/v3/projects/38b06ef0c29244659a14cf85f67345dc` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:14.860827 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d382f6-d75b-4654-82d0-c6f172abbd7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=38b06ef0c29244659a14cf85f67345dc)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:14.862778 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d382f6-d75b-4654-82d0-c6f172abbd7a admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:14.912517 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e8d382f6-d75b-4654-82d0-c6f172abbd7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38b06ef0c29244659a14cf85f67345dc', 'request_id': 'req-e8d382f6-d75b-4654-82d0-c6f172abbd7a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:14.920912 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e8d382f6-d75b-4654-82d0-c6f172abbd7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38b06ef0c29244659a14cf85f67345dc', 'request_id': 'req-e8d382f6-d75b-4654-82d0-c6f172abbd7a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:14.929966 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e8d382f6-d75b-4654-82d0-c6f172abbd7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38b06ef0c29244659a14cf85f67345dc', 'request_id': 'req-e8d382f6-d75b-4654-82d0-c6f172abbd7a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:14.942022 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e8d382f6-d75b-4654-82d0-c6f172abbd7a admin admin] The token cache is being invalidate because project 38b06ef0c29244659a14cf85f67345dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:45:14.942341 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e8d382f6-d75b-4654-82d0-c6f172abbd7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:14.942998 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e8d382f6-d75b-4654-82d0-c6f172abbd7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:14.944776 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1297/2598] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:45:14 2022] DELETE /identity/v3/projects/38b06ef0c29244659a14cf85f67345dc => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:45:14.948658 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5283c11-058b-424b-a78b-12a0bddf7747 service neutron] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:14.959641 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5283c11-058b-424b-a78b-12a0bddf7747 service neutron] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:15.022012 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be93ad9-4a75-4ee6-9a7f-4f360f8c5e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:15.022926 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be93ad9-4a75-4ee6-9a7f-4f360f8c5e4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:15.023218 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be93ad9-4a75-4ee6-9a7f-4f360f8c5e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:15.023502 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be93ad9-4a75-4ee6-9a7f-4f360f8c5e4b admin admin] PATH_INFO: `/v3/users/0895090e443a41afbe0a41be2076ffe2` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:15.024715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be93ad9-4a75-4ee6-9a7f-4f360f8c5e4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0895090e443a41afbe0a41be2076ffe2)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:15.026670 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be93ad9-4a75-4ee6-9a7f-4f360f8c5e4b admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:15.076256 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-2be93ad9-4a75-4ee6-9a7f-4f360f8c5e4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0895090e443a41afbe0a41be2076ffe2'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:15.076935 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-2be93ad9-4a75-4ee6-9a7f-4f360f8c5e4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0895090e443a41afbe0a41be2076ffe2'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:15.085441 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-2be93ad9-4a75-4ee6-9a7f-4f360f8c5e4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0895090e443a41afbe0a41be2076ffe2'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:15.086224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-2be93ad9-4a75-4ee6-9a7f-4f360f8c5e4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0895090e443a41afbe0a41be2076ffe2'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:15.094030 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-2be93ad9-4a75-4ee6-9a7f-4f360f8c5e4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0895090e443a41afbe0a41be2076ffe2'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:15.122424 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1302/2599] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:45:14 2022] DELETE /identity/v3/users/0895090e443a41afbe0a41be2076ffe2 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:45:15.602847 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d382f6-d75b-4654-82d0-c6f172abbd7a admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:15.604582 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e8d382f6-d75b-4654-82d0-c6f172abbd7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:15.621511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d382f6-d75b-4654-82d0-c6f172abbd7a admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:15.701107 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbcc265-4722-4af4-b7be-ae3d12e81019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:15.701988 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcc265-4722-4af4-b7be-ae3d12e81019 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:15.702321 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcc265-4722-4af4-b7be-ae3d12e81019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:15.702606 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcc265-4722-4af4-b7be-ae3d12e81019 admin admin] PATH_INFO: `/v3/projects/04e6fbe2dc2b4608a62f4ebe4db882a2` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:15.703845 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbcc265-4722-4af4-b7be-ae3d12e81019 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04e6fbe2dc2b4608a62f4ebe4db882a2)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:15.705788 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbcc265-4722-4af4-b7be-ae3d12e81019 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:15.712365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:15.757946 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2cbcc265-4722-4af4-b7be-ae3d12e81019 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04e6fbe2dc2b4608a62f4ebe4db882a2', 'request_id': 'req-2cbcc265-4722-4af4-b7be-ae3d12e81019'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:15.765973 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2cbcc265-4722-4af4-b7be-ae3d12e81019 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04e6fbe2dc2b4608a62f4ebe4db882a2', 'request_id': 'req-2cbcc265-4722-4af4-b7be-ae3d12e81019'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:15.773108 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2cbcc265-4722-4af4-b7be-ae3d12e81019 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04e6fbe2dc2b4608a62f4ebe4db882a2', 'request_id': 'req-2cbcc265-4722-4af4-b7be-ae3d12e81019'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:15.784473 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2cbcc265-4722-4af4-b7be-ae3d12e81019 admin admin] The token cache is being invalidate because project 04e6fbe2dc2b4608a62f4ebe4db882a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:45:15.784778 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2cbcc265-4722-4af4-b7be-ae3d12e81019 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:15.785410 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2cbcc265-4722-4af4-b7be-ae3d12e81019 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:15.787066 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1298/2600] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:45:15 2022] DELETE /identity/v3/projects/04e6fbe2dc2b4608a62f4ebe4db882a2 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:45:15.791986 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae2f8a-a47f-48b1-9c04-748a89d9fb6c None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:15.792467 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae2f8a-a47f-48b1-9c04-748a89d9fb6c None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:15.792972 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae2f8a-a47f-48b1-9c04-748a89d9fb6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:15.811387 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-eeae2f8a-a47f-48b1-9c04-748a89d9fb6c None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:45:15.812610 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-eeae2f8a-a47f-48b1-9c04-748a89d9fb6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:15.826490 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:15.878897 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1303/2601] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:45:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:45:15.887187 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad2ab3-0ff1-4878-886a-e45bf228f405 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:15.887569 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad2ab3-0ff1-4878-886a-e45bf228f405 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:15.889007 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad2ab3-0ff1-4878-886a-e45bf228f405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:15.908175 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c1ad2ab3-0ff1-4878-886a-e45bf228f405 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:45:15.909401 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c1ad2ab3-0ff1-4878-886a-e45bf228f405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:15.917566 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1299/2602] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:45:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:45:15.921920 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae2f8a-a47f-48b1-9c04-748a89d9fb6c None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:15.932009 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae2f8a-a47f-48b1-9c04-748a89d9fb6c None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:15.938676 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec2b07-a43c-45dc-a41b-0701439b8023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:15.939544 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec2b07-a43c-45dc-a41b-0701439b8023 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:15.939831 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec2b07-a43c-45dc-a41b-0701439b8023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:15.940106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec2b07-a43c-45dc-a41b-0701439b8023 admin admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:15.940841 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec2b07-a43c-45dc-a41b-0701439b8023 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:45:15.941147 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec2b07-a43c-45dc-a41b-0701439b8023 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:15.942972 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec2b07-a43c-45dc-a41b-0701439b8023 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:15.957760 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1304/2603] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:45:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:45:15.962015 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad2ab3-0ff1-4878-886a-e45bf228f405 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:15.963982 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c1ad2ab3-0ff1-4878-886a-e45bf228f405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:15.964622 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec2b07-a43c-45dc-a41b-0701439b8023 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:15.979894 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad2ab3-0ff1-4878-886a-e45bf228f405 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:15.981080 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec2b07-a43c-45dc-a41b-0701439b8023 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:15.985074 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860cc8d1-33e4-4a6e-b833-188f63e1a2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:15.986049 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cc8d1-33e4-4a6e-b833-188f63e1a2fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:15.986494 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ebaee-d7af-4242-a788-7c54015123d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:15.986673 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cc8d1-33e4-4a6e-b833-188f63e1a2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:15.986998 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cc8d1-33e4-4a6e-b833-188f63e1a2fe admin admin] PATH_INFO: `/v3/users/156daafe58fd4e248020b225b9c2c210` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:15.987155 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ebaee-d7af-4242-a788-7c54015123d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:15.987416 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ebaee-d7af-4242-a788-7c54015123d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:15.987740 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ebaee-d7af-4242-a788-7c54015123d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:15.988166 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860cc8d1-33e4-4a6e-b833-188f63e1a2fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=156daafe58fd4e248020b225b9c2c210)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:15.988431 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ebaee-d7af-4242-a788-7c54015123d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:15.990197 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860cc8d1-33e4-4a6e-b833-188f63e1a2fe admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:15.990372 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ebaee-d7af-4242-a788-7c54015123d9 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:16.018733 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1305/2604] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:45:15 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:45:16.022627 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ebaee-d7af-4242-a788-7c54015123d9 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:16.043495 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-860cc8d1-33e4-4a6e-b833-188f63e1a2fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '156daafe58fd4e248020b225b9c2c210'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:16.044798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-860cc8d1-33e4-4a6e-b833-188f63e1a2fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '156daafe58fd4e248020b225b9c2c210'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:16.045446 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-860cc8d1-33e4-4a6e-b833-188f63e1a2fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '156daafe58fd4e248020b225b9c2c210'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:16.057342 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-860cc8d1-33e4-4a6e-b833-188f63e1a2fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '156daafe58fd4e248020b225b9c2c210'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:16.065539 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-860cc8d1-33e4-4a6e-b833-188f63e1a2fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '156daafe58fd4e248020b225b9c2c210'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:16.078537 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1300/2605] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:45:15 2022] DELETE /identity/v3/users/156daafe58fd4e248020b225b9c2c210 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:45:16.090807 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ebaee-d7af-4242-a788-7c54015123d9 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:16.148402 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0986296f-60cc-4b1a-8fd4-f5aca8482a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:16.149083 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986296f-60cc-4b1a-8fd4-f5aca8482a32 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:16.149425 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986296f-60cc-4b1a-8fd4-f5aca8482a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:16.149710 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986296f-60cc-4b1a-8fd4-f5aca8482a32 admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:16.150458 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0986296f-60cc-4b1a-8fd4-f5aca8482a32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:16.152355 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0986296f-60cc-4b1a-8fd4-f5aca8482a32 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:16.177429 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1306/2606] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:45:16 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:45:16.178740 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860cc8d1-33e4-4a6e-b833-188f63e1a2fe admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:16.180882 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-860cc8d1-33e4-4a6e-b833-188f63e1a2fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:16.195864 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860cc8d1-33e4-4a6e-b833-188f63e1a2fe admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:16.200750 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52a5b2-70e4-43ec-91bf-79c89fb11daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:16.201515 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52a5b2-70e4-43ec-91bf-79c89fb11daa admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:16.201834 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52a5b2-70e4-43ec-91bf-79c89fb11daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:16.202289 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52a5b2-70e4-43ec-91bf-79c89fb11daa admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:16.203096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de52a5b2-70e4-43ec-91bf-79c89fb11daa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:16.205706 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de52a5b2-70e4-43ec-91bf-79c89fb11daa admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:16.217670 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1301/2607] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:45:16.221677 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0986296f-60cc-4b1a-8fd4-f5aca8482a32 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:16.232313 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0986296f-60cc-4b1a-8fd4-f5aca8482a32 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:16.237144 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4502036d-787f-4f98-bdc5-873f19880630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:16.237837 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502036d-787f-4f98-bdc5-873f19880630 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:16.238149 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502036d-787f-4f98-bdc5-873f19880630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:16.238434 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502036d-787f-4f98-bdc5-873f19880630 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:16.239172 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4502036d-787f-4f98-bdc5-873f19880630 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:16.240772 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4502036d-787f-4f98-bdc5-873f19880630 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:16.256000 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1307/2608] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:45:16.257192 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52a5b2-70e4-43ec-91bf-79c89fb11daa admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:16.261223 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4502036d-787f-4f98-bdc5-873f19880630 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:16.263500 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-4502036d-787f-4f98-bdc5-873f19880630 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:16.268020 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52a5b2-70e4-43ec-91bf-79c89fb11daa admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:16.272747 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e759ae3-5d8b-438f-9f22-08115e540ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:16.273637 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e759ae3-5d8b-438f-9f22-08115e540ed4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:16.273977 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e759ae3-5d8b-438f-9f22-08115e540ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:16.274217 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e759ae3-5d8b-438f-9f22-08115e540ed4 admin admin] PATH_INFO: `/v3/projects/203972aef723487d950c600501303a94/users/8f93a3d32f6c4151be0d7493c693c377/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:16.277687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4502036d-787f-4f98-bdc5-873f19880630 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:16.282482 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ec732-cbe1-4aef-ba67-7fc757bb2964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:16.283347 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ec732-cbe1-4aef-ba67-7fc757bb2964 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:16.283630 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ec732-cbe1-4aef-ba67-7fc757bb2964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:16.283905 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ec732-cbe1-4aef-ba67-7fc757bb2964 admin admin] PATH_INFO: `/v3/projects/cbaf2f777f01437ca5891860365e08ce` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:16.285106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481ec732-cbe1-4aef-ba67-7fc757bb2964 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbaf2f777f01437ca5891860365e08ce)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:16.287098 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481ec732-cbe1-4aef-ba67-7fc757bb2964 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:16.295321 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:16.304305 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e759ae3-5d8b-438f-9f22-08115e540ed4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=203972aef723487d950c600501303a94, user_id=8f93a3d32f6c4151be0d7493c693c377, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:16.307326 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e759ae3-5d8b-438f-9f22-08115e540ed4 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:16.322345 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1302/2609] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:45:16 2022] PUT /identity/v3/projects/203972aef723487d950c600501303a94/users/8f93a3d32f6c4151be0d7493c693c377/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:45:16.333655 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e43dc1-fd21-4734-abfe-9d7bbc20d4c1 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:16.333949 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e43dc1-fd21-4734-abfe-9d7bbc20d4c1 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:16.334252 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e43dc1-fd21-4734-abfe-9d7bbc20d4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:16.342653 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-481ec732-cbe1-4aef-ba67-7fc757bb2964 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbaf2f777f01437ca5891860365e08ce', 'request_id': 'req-481ec732-cbe1-4aef-ba67-7fc757bb2964'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:16.351012 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-481ec732-cbe1-4aef-ba67-7fc757bb2964 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbaf2f777f01437ca5891860365e08ce', 'request_id': 'req-481ec732-cbe1-4aef-ba67-7fc757bb2964'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:16.358463 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-481ec732-cbe1-4aef-ba67-7fc757bb2964 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbaf2f777f01437ca5891860365e08ce', 'request_id': 'req-481ec732-cbe1-4aef-ba67-7fc757bb2964'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:16.362979 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-55e43dc1-fd21-4734-abfe-9d7bbc20d4c1 None None] MFA Rules not processed for user `8f93a3d32f6c4151be0d7493c693c377`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:45:16.364197 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-55e43dc1-fd21-4734-abfe-9d7bbc20d4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:16.371164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-481ec732-cbe1-4aef-ba67-7fc757bb2964 admin admin] The token cache is being invalidate because project cbaf2f777f01437ca5891860365e08ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:45:16.371455 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-481ec732-cbe1-4aef-ba67-7fc757bb2964 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:16.372731 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-481ec732-cbe1-4aef-ba67-7fc757bb2964 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:16.374363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1308/2610] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:45:16 2022] DELETE /identity/v3/projects/cbaf2f777f01437ca5891860365e08ce => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:45:16.378236 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ec732-cbe1-4aef-ba67-7fc757bb2964 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:16.379582 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-481ec732-cbe1-4aef-ba67-7fc757bb2964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:16.380112 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:16.435797 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1303/2611] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:45:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:45:16.440164 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e43dc1-fd21-4734-abfe-9d7bbc20d4c1 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:16.441469 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-55e43dc1-fd21-4734-abfe-9d7bbc20d4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:16.446345 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ec732-cbe1-4aef-ba67-7fc757bb2964 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:16.451087 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76875ec1-d259-4f7a-8ee3-83ee509e87e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:16.452000 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76875ec1-d259-4f7a-8ee3-83ee509e87e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:16.452288 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76875ec1-d259-4f7a-8ee3-83ee509e87e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:16.452576 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76875ec1-d259-4f7a-8ee3-83ee509e87e8 admin admin] PATH_INFO: `/v3/users/ccf4b7d559644c6887dbeac50812df74` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:16.453795 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76875ec1-d259-4f7a-8ee3-83ee509e87e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccf4b7d559644c6887dbeac50812df74)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:16.454487 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e43dc1-fd21-4734-abfe-9d7bbc20d4c1 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:16.455759 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76875ec1-d259-4f7a-8ee3-83ee509e87e8 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:16.459131 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f04f5e-6b8a-4f40-9c4a-62269aa6c246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:16.459780 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f04f5e-6b8a-4f40-9c4a-62269aa6c246 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:16.460067 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f04f5e-6b8a-4f40-9c4a-62269aa6c246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:16.460342 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f04f5e-6b8a-4f40-9c4a-62269aa6c246 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:16.461039 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f04f5e-6b8a-4f40-9c4a-62269aa6c246 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:16.462962 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f04f5e-6b8a-4f40-9c4a-62269aa6c246 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:16.491495 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1304/2612] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:45:16 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:45:16.495828 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f04f5e-6b8a-4f40-9c4a-62269aa6c246 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:16.506336 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-76875ec1-d259-4f7a-8ee3-83ee509e87e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccf4b7d559644c6887dbeac50812df74'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:16.507396 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-76875ec1-d259-4f7a-8ee3-83ee509e87e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccf4b7d559644c6887dbeac50812df74'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:16.515643 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-76875ec1-d259-4f7a-8ee3-83ee509e87e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccf4b7d559644c6887dbeac50812df74'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:16.516194 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-76875ec1-d259-4f7a-8ee3-83ee509e87e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccf4b7d559644c6887dbeac50812df74'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:16.524718 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-76875ec1-d259-4f7a-8ee3-83ee509e87e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccf4b7d559644c6887dbeac50812df74'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:16.547281 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1309/2613] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:45:16 2022] DELETE /identity/v3/users/ccf4b7d559644c6887dbeac50812df74 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:45:16.564188 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f04f5e-6b8a-4f40-9c4a-62269aa6c246 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:16.627392 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45145bd-346a-4c45-ad1b-f9276fd119fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:16.628706 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45145bd-346a-4c45-ad1b-f9276fd119fa admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:16.628991 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45145bd-346a-4c45-ad1b-f9276fd119fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:16.629318 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45145bd-346a-4c45-ad1b-f9276fd119fa admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:16.630344 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45145bd-346a-4c45-ad1b-f9276fd119fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:16.632545 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45145bd-346a-4c45-ad1b-f9276fd119fa admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:16.655819 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1305/2614] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:45:16 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:45:16.659749 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76875ec1-d259-4f7a-8ee3-83ee509e87e8 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:16.661195 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-76875ec1-d259-4f7a-8ee3-83ee509e87e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:16.676826 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76875ec1-d259-4f7a-8ee3-83ee509e87e8 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:16.682884 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a880c5fb-fc1b-41f0-abeb-fca5a047f152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:16.684426 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880c5fb-fc1b-41f0-abeb-fca5a047f152 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:16.684426 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880c5fb-fc1b-41f0-abeb-fca5a047f152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:16.684750 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880c5fb-fc1b-41f0-abeb-fca5a047f152 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:16.685466 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a880c5fb-fc1b-41f0-abeb-fca5a047f152 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:16.698341 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a880c5fb-fc1b-41f0-abeb-fca5a047f152 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:16.699644 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1310/2615] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:45:16.704755 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45145bd-346a-4c45-ad1b-f9276fd119fa admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:16.716427 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45145bd-346a-4c45-ad1b-f9276fd119fa admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:16.721992 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff108f2f-55c7-4b39-8cd8-7d77a680f998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:16.723057 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff108f2f-55c7-4b39-8cd8-7d77a680f998 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:16.723363 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff108f2f-55c7-4b39-8cd8-7d77a680f998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:16.723626 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff108f2f-55c7-4b39-8cd8-7d77a680f998 admin admin] PATH_INFO: `/v3/projects/9257d5d0e871496e804457058584953c/users/2e4f4e49453c44cf907a654590803362/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:16.730490 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a880c5fb-fc1b-41f0-abeb-fca5a047f152 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:16.733026 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-a880c5fb-fc1b-41f0-abeb-fca5a047f152 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:16.744863 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:16.745804 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff108f2f-55c7-4b39-8cd8-7d77a680f998 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9257d5d0e871496e804457058584953c, user_id=2e4f4e49453c44cf907a654590803362, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:16.750063 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff108f2f-55c7-4b39-8cd8-7d77a680f998 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:16.753198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a880c5fb-fc1b-41f0-abeb-fca5a047f152 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:16.761415 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ace3f13-2f25-4543-bec9-036ff1520e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:16.765294 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace3f13-2f25-4543-bec9-036ff1520e12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:16.765869 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace3f13-2f25-4543-bec9-036ff1520e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:16.766512 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace3f13-2f25-4543-bec9-036ff1520e12 admin admin] PATH_INFO: `/v3/projects/b2d031f1a6354c9a9722aefa5cfe72f4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:16.770504 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ace3f13-2f25-4543-bec9-036ff1520e12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2d031f1a6354c9a9722aefa5cfe72f4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:16.771390 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1306/2616] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:45:16 2022] PUT /identity/v3/projects/9257d5d0e871496e804457058584953c/users/2e4f4e49453c44cf907a654590803362/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:45:16.773513 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ace3f13-2f25-4543-bec9-036ff1520e12 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:16.775725 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff108f2f-55c7-4b39-8cd8-7d77a680f998 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:16.786290 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:16.840025 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-9ace3f13-2f25-4543-bec9-036ff1520e12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2d031f1a6354c9a9722aefa5cfe72f4', 'request_id': 'req-9ace3f13-2f25-4543-bec9-036ff1520e12'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:16.851000 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-9ace3f13-2f25-4543-bec9-036ff1520e12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2d031f1a6354c9a9722aefa5cfe72f4', 'request_id': 'req-9ace3f13-2f25-4543-bec9-036ff1520e12'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:16.853592 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff108f2f-55c7-4b39-8cd8-7d77a680f998 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:16.858889 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-9ace3f13-2f25-4543-bec9-036ff1520e12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2d031f1a6354c9a9722aefa5cfe72f4', 'request_id': 'req-9ace3f13-2f25-4543-bec9-036ff1520e12'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:16.873850 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-9ace3f13-2f25-4543-bec9-036ff1520e12 admin admin] The token cache is being invalidate because project b2d031f1a6354c9a9722aefa5cfe72f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:45:16.874167 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-9ace3f13-2f25-4543-bec9-036ff1520e12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:16.874841 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-9ace3f13-2f25-4543-bec9-036ff1520e12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:16.882158 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1311/2617] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:45:16 2022] DELETE /identity/v3/projects/b2d031f1a6354c9a9722aefa5cfe72f4 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:45:16.883955 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e0c74-51e0-44fa-b704-d32e9a3231e9 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:16.884429 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e0c74-51e0-44fa-b704-d32e9a3231e9 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:16.884959 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e0c74-51e0-44fa-b704-d32e9a3231e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:16.908412 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-c08e0c74-51e0-44fa-b704-d32e9a3231e9 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:45:16.909815 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-c08e0c74-51e0-44fa-b704-d32e9a3231e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:16.928063 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff0f85a-a669-4ccc-b345-76a4b461c048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:16.928742 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0f85a-a669-4ccc-b345-76a4b461c048 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:16.929388 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0f85a-a669-4ccc-b345-76a4b461c048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:16.929388 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0f85a-a669-4ccc-b345-76a4b461c048 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:16.930031 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff0f85a-a669-4ccc-b345-76a4b461c048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:16.931749 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff0f85a-a669-4ccc-b345-76a4b461c048 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:16.942790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1307/2618] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:45:16.946793 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff0f85a-a669-4ccc-b345-76a4b461c048 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:16.948699 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-5ff0f85a-a669-4ccc-b345-76a4b461c048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:16.963049 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff0f85a-a669-4ccc-b345-76a4b461c048 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:16.969622 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760b24cb-10d1-4d9f-979a-702e697c70e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:16.970737 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b24cb-10d1-4d9f-979a-702e697c70e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:16.971024 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b24cb-10d1-4d9f-979a-702e697c70e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:16.971306 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b24cb-10d1-4d9f-979a-702e697c70e5 admin admin] PATH_INFO: `/v3/projects/9257d5d0e871496e804457058584953c/users/2e4f4e49453c44cf907a654590803362/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:16.974111 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760b24cb-10d1-4d9f-979a-702e697c70e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9257d5d0e871496e804457058584953c, user_id=2e4f4e49453c44cf907a654590803362, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:16.977685 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760b24cb-10d1-4d9f-979a-702e697c70e5 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:16.991235 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1312/2619] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:45:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:45:16.994589 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1308/2620] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:45:16 2022] PUT /identity/v3/projects/9257d5d0e871496e804457058584953c/users/2e4f4e49453c44cf907a654590803362/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:45:16.999987 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501f406-e35a-4cf1-82dc-3731f7b7fa45 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:17.000277 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501f406-e35a-4cf1-82dc-3731f7b7fa45 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:17.000553 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501f406-e35a-4cf1-82dc-3731f7b7fa45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:17.006055 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d798f-5dc5-489b-b37c-008eeb2f66ab None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:17.006437 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d798f-5dc5-489b-b37c-008eeb2f66ab None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:17.006657 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d798f-5dc5-489b-b37c-008eeb2f66ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:17.021918 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-f501f406-e35a-4cf1-82dc-3731f7b7fa45 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:45:17.023334 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-f501f406-e35a-4cf1-82dc-3731f7b7fa45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:17.045832 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-333d798f-5dc5-489b-b37c-008eeb2f66ab None None] MFA Rules not processed for user `2e4f4e49453c44cf907a654590803362`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:45:17.047181 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-333d798f-5dc5-489b-b37c-008eeb2f66ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:17.085332 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1313/2621] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:45:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:45:17.089719 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f501f406-e35a-4cf1-82dc-3731f7b7fa45 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:17.100857 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f501f406-e35a-4cf1-82dc-3731f7b7fa45 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:17.105995 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d53df-b81f-479f-854b-e9e24a968fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:17.107107 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d53df-b81f-479f-854b-e9e24a968fe2 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:17.107107 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d53df-b81f-479f-854b-e9e24a968fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:17.107404 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d53df-b81f-479f-854b-e9e24a968fe2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:17.108119 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5d53df-b81f-479f-854b-e9e24a968fe2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:45:17.108430 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5d53df-b81f-479f-854b-e9e24a968fe2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:17.110112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5d53df-b81f-479f-854b-e9e24a968fe2 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:17.127636 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1314/2622] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:45:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:45:17.127830 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1309/2623] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:45:17 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:45:17.128033 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d53df-b81f-479f-854b-e9e24a968fe2 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:17.135300 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d798f-5dc5-489b-b37c-008eeb2f66ab None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:17.136793 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-333d798f-5dc5-489b-b37c-008eeb2f66ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:17.139405 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d53df-b81f-479f-854b-e9e24a968fe2 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:17.144777 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc1728-0815-41d2-a0e0-2ffc048c491d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:17.145429 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc1728-0815-41d2-a0e0-2ffc048c491d admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:17.145711 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc1728-0815-41d2-a0e0-2ffc048c491d admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:17.145982 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc1728-0815-41d2-a0e0-2ffc048c491d admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:17.146768 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfc1728-0815-41d2-a0e0-2ffc048c491d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:17.149060 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfc1728-0815-41d2-a0e0-2ffc048c491d admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:17.161278 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:17.161496 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:17.181531 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1315/2624] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:45:17 2022] POST /identity/v3/projects => generated 369 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:45:17.186297 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc1728-0815-41d2-a0e0-2ffc048c491d admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:17.213779 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d798f-5dc5-489b-b37c-008eeb2f66ab None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:17.252484 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc1728-0815-41d2-a0e0-2ffc048c491d admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:17.257133 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53599a-0496-4823-96a9-5fda521acdb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:17.257797 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53599a-0496-4823-96a9-5fda521acdb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:17.258080 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53599a-0496-4823-96a9-5fda521acdb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:17.258386 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53599a-0496-4823-96a9-5fda521acdb8 admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:17.259101 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53599a-0496-4823-96a9-5fda521acdb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:17.261028 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53599a-0496-4823-96a9-5fda521acdb8 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:17.266476 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18627ef-2375-4d6e-9229-ca4c2588350f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:17.267127 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18627ef-2375-4d6e-9229-ca4c2588350f service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:17.267415 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18627ef-2375-4d6e-9229-ca4c2588350f service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:17.267695 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18627ef-2375-4d6e-9229-ca4c2588350f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:17.286144 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1316/2625] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:45:17 2022] POST /identity/v3/users => generated 285 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:45:17.288157 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53599a-0496-4823-96a9-5fda521acdb8 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:17.298513 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53599a-0496-4823-96a9-5fda521acdb8 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:17.303556 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a294ea-39a5-448b-a24f-0ff86eb454bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:17.304433 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a294ea-39a5-448b-a24f-0ff86eb454bc admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:17.304717 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a294ea-39a5-448b-a24f-0ff86eb454bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:17.304990 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a294ea-39a5-448b-a24f-0ff86eb454bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:17.305726 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a294ea-39a5-448b-a24f-0ff86eb454bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:17.307363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a294ea-39a5-448b-a24f-0ff86eb454bc admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:17.319666 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1317/2626] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:45:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:45:17.323914 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a294ea-39a5-448b-a24f-0ff86eb454bc admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:17.337596 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18627ef-2375-4d6e-9229-ca4c2588350f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:17.339797 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18627ef-2375-4d6e-9229-ca4c2588350f service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:17.343652 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a294ea-39a5-448b-a24f-0ff86eb454bc admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:17.348710 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24191846-3559-47ee-92a4-0d9bd7e3e666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:17.349494 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24191846-3559-47ee-92a4-0d9bd7e3e666 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:17.349780 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24191846-3559-47ee-92a4-0d9bd7e3e666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:17.350059 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24191846-3559-47ee-92a4-0d9bd7e3e666 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:17.350816 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24191846-3559-47ee-92a4-0d9bd7e3e666 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:17.351265 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1310/2627] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:45:17 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:45:17.353173 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24191846-3559-47ee-92a4-0d9bd7e3e666 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:17.365544 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1318/2628] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:45:17 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:45:17.370430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18627ef-2375-4d6e-9229-ca4c2588350f service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:17.386220 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18627ef-2375-4d6e-9229-ca4c2588350f service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:17.386748 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70559663-8cc9-4ff4-8d4e-77dddb35de14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:17.388373 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70559663-8cc9-4ff4-8d4e-77dddb35de14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:17.388486 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70559663-8cc9-4ff4-8d4e-77dddb35de14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:17.388759 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70559663-8cc9-4ff4-8d4e-77dddb35de14 admin admin] PATH_INFO: `/v3/projects/f6b10aa1658d47d0b8217a7013d67950/users/652e58dff72d437ea9b90fd096ca765c/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:17.405084 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70559663-8cc9-4ff4-8d4e-77dddb35de14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6b10aa1658d47d0b8217a7013d67950, user_id=652e58dff72d437ea9b90fd096ca765c, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:17.408007 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70559663-8cc9-4ff4-8d4e-77dddb35de14 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:17.423192 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1311/2629] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:45:17 2022] PUT /identity/v3/projects/f6b10aa1658d47d0b8217a7013d67950/users/652e58dff72d437ea9b90fd096ca765c/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:45:17.425817 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24191846-3559-47ee-92a4-0d9bd7e3e666 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:17.433287 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da2198-35a6-4f23-ae94-facbe01f6728 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:17.433572 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da2198-35a6-4f23-ae94-facbe01f6728 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:17.433881 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da2198-35a6-4f23-ae94-facbe01f6728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:17.436829 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24191846-3559-47ee-92a4-0d9bd7e3e666 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:17.462537 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-52da2198-35a6-4f23-ae94-facbe01f6728 None None] MFA Rules not processed for user `652e58dff72d437ea9b90fd096ca765c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:45:17.464324 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-52da2198-35a6-4f23-ae94-facbe01f6728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:17.486963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf99d366-408b-4be2-a5bd-97aaf36d8073 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:17.487623 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99d366-408b-4be2-a5bd-97aaf36d8073 service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:17.487907 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99d366-408b-4be2-a5bd-97aaf36d8073 service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:17.488182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99d366-408b-4be2-a5bd-97aaf36d8073 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:17.489956 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-cf99d366-408b-4be2-a5bd-97aaf36d8073 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:17.514674 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:17.532533 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1312/2630] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:45:17 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:45:17.544438 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52da2198-35a6-4f23-ae94-facbe01f6728 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:17.545847 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-52da2198-35a6-4f23-ae94-facbe01f6728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:17.554116 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf99d366-408b-4be2-a5bd-97aaf36d8073 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:17.555653 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf99d366-408b-4be2-a5bd-97aaf36d8073 service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:17.565361 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1319/2631] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:45:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:45:17.626313 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52da2198-35a6-4f23-ae94-facbe01f6728 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:17.630327 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d60db-1a3c-469a-b453-9a9e3e4c2f0d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfca4d302aa43249609513f35d55d7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:17.631111 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d60db-1a3c-469a-b453-9a9e3e4c2f0d service glance] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:17.631325 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d60db-1a3c-469a-b453-9a9e3e4c2f0d service glance] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:17.631590 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d60db-1a3c-469a-b453-9a9e3e4c2f0d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:17.640521 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989d60db-1a3c-469a-b453-9a9e3e4c2f0d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:17.641955 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989d60db-1a3c-469a-b453-9a9e3e4c2f0d service glance] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:17.653353 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1313/2632] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:45:17 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:45:18.112499 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf99d366-408b-4be2-a5bd-97aaf36d8073 service cinder] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:18.113899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-cf99d366-408b-4be2-a5bd-97aaf36d8073 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:18.128980 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:18.184488 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf99d366-408b-4be2-a5bd-97aaf36d8073 service cinder] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:18.189005 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e19f07-2ebe-4a9e-a2c9-fc8e3fe8df92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:18.189875 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e19f07-2ebe-4a9e-a2c9-fc8e3fe8df92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:18.190190 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e19f07-2ebe-4a9e-a2c9-fc8e3fe8df92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:18.190466 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e19f07-2ebe-4a9e-a2c9-fc8e3fe8df92 admin admin] PATH_INFO: `/v3/users/8e8153a1c4e54cfcbecf8a9bac4ade86` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:18.191655 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e19f07-2ebe-4a9e-a2c9-fc8e3fe8df92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e8153a1c4e54cfcbecf8a9bac4ade86)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:18.193558 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e19f07-2ebe-4a9e-a2c9-fc8e3fe8df92 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:18.241002 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-47e19f07-2ebe-4a9e-a2c9-fc8e3fe8df92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e8153a1c4e54cfcbecf8a9bac4ade86'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:18.241660 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-47e19f07-2ebe-4a9e-a2c9-fc8e3fe8df92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e8153a1c4e54cfcbecf8a9bac4ade86'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:18.249689 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-47e19f07-2ebe-4a9e-a2c9-fc8e3fe8df92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e8153a1c4e54cfcbecf8a9bac4ade86'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:18.250377 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-47e19f07-2ebe-4a9e-a2c9-fc8e3fe8df92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e8153a1c4e54cfcbecf8a9bac4ade86'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:18.257879 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-47e19f07-2ebe-4a9e-a2c9-fc8e3fe8df92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e8153a1c4e54cfcbecf8a9bac4ade86'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:18.276594 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1320/2633] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:45:18 2022] DELETE /identity/v3/users/8e8153a1c4e54cfcbecf8a9bac4ade86 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:45:18.282762 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3083e535-58b8-4159-b1fa-e812c911522a None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:18.283126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3083e535-58b8-4159-b1fa-e812c911522a None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:18.283475 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3083e535-58b8-4159-b1fa-e812c911522a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:18.301866 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-3083e535-58b8-4159-b1fa-e812c911522a None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:45:18.303203 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3083e535-58b8-4159-b1fa-e812c911522a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:18.367885 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1314/2634] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:45:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:45:18.378636 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e19f07-2ebe-4a9e-a2c9-fc8e3fe8df92 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:18.380055 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-47e19f07-2ebe-4a9e-a2c9-fc8e3fe8df92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:18.443076 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e19f07-2ebe-4a9e-a2c9-fc8e3fe8df92 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:18.446926 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc091f-5ecf-4846-b96c-66bee69d0da9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:18.447588 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc091f-5ecf-4846-b96c-66bee69d0da9 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:18.447876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc091f-5ecf-4846-b96c-66bee69d0da9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:18.448155 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc091f-5ecf-4846-b96c-66bee69d0da9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:18.459320 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bc091f-5ecf-4846-b96c-66bee69d0da9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:18.460807 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bc091f-5ecf-4846-b96c-66bee69d0da9 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:18.471734 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1321/2635] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:45:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:45:18.758333 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3083e535-58b8-4159-b1fa-e812c911522a None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:18.759916 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3083e535-58b8-4159-b1fa-e812c911522a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:18.777573 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3083e535-58b8-4159-b1fa-e812c911522a None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:18.783466 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53188f36-c7d2-48af-b591-2975de15964c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:18.784351 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53188f36-c7d2-48af-b591-2975de15964c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:18.784809 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53188f36-c7d2-48af-b591-2975de15964c admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:18.785089 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53188f36-c7d2-48af-b591-2975de15964c admin admin] PATH_INFO: `/v3/projects/98d1f507630f433a88495bac1b3235a4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:18.786372 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53188f36-c7d2-48af-b591-2975de15964c admin admin] RBAC: Authorizing `identity:delete_project(project_id=98d1f507630f433a88495bac1b3235a4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:18.788450 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53188f36-c7d2-48af-b591-2975de15964c admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:18.841333 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-53188f36-c7d2-48af-b591-2975de15964c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98d1f507630f433a88495bac1b3235a4', 'request_id': 'req-53188f36-c7d2-48af-b591-2975de15964c'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:18.849830 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-53188f36-c7d2-48af-b591-2975de15964c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98d1f507630f433a88495bac1b3235a4', 'request_id': 'req-53188f36-c7d2-48af-b591-2975de15964c'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:18.856541 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-53188f36-c7d2-48af-b591-2975de15964c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98d1f507630f433a88495bac1b3235a4', 'request_id': 'req-53188f36-c7d2-48af-b591-2975de15964c'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:18.863910 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:18.868881 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-53188f36-c7d2-48af-b591-2975de15964c admin admin] The token cache is being invalidate because project 98d1f507630f433a88495bac1b3235a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:45:18.869409 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-53188f36-c7d2-48af-b591-2975de15964c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:18.871394 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-53188f36-c7d2-48af-b591-2975de15964c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:18.874644 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1315/2636] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:45:18 2022] DELETE /identity/v3/projects/98d1f507630f433a88495bac1b3235a4 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:45:18.878704 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc091f-5ecf-4846-b96c-66bee69d0da9 service neutron] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:18.891325 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc091f-5ecf-4846-b96c-66bee69d0da9 service neutron] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:18.931138 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:18.961101 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e2ba0c-ef96-45fe-b094-68c1f2c7c1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:18.962396 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2ba0c-ef96-45fe-b094-68c1f2c7c1b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:18.962686 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2ba0c-ef96-45fe-b094-68c1f2c7c1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:18.963038 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2ba0c-ef96-45fe-b094-68c1f2c7c1b7 admin admin] PATH_INFO: `/v3/users/685b927bda6c4c319ec7a30773f27b1a` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:18.966596 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e2ba0c-ef96-45fe-b094-68c1f2c7c1b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=685b927bda6c4c319ec7a30773f27b1a)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:18.967148 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e2ba0c-ef96-45fe-b094-68c1f2c7c1b7 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:19.024898 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-b0e2ba0c-ef96-45fe-b094-68c1f2c7c1b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '685b927bda6c4c319ec7a30773f27b1a'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:19.025901 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-b0e2ba0c-ef96-45fe-b094-68c1f2c7c1b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '685b927bda6c4c319ec7a30773f27b1a'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:19.034882 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-b0e2ba0c-ef96-45fe-b094-68c1f2c7c1b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '685b927bda6c4c319ec7a30773f27b1a'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:19.035783 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-b0e2ba0c-ef96-45fe-b094-68c1f2c7c1b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '685b927bda6c4c319ec7a30773f27b1a'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:19.050549 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-b0e2ba0c-ef96-45fe-b094-68c1f2c7c1b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '685b927bda6c4c319ec7a30773f27b1a'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:19.083707 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1322/2637] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:45:18 2022] DELETE /identity/v3/users/685b927bda6c4c319ec7a30773f27b1a => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:45:19.373836 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53188f36-c7d2-48af-b591-2975de15964c admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:19.375384 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-53188f36-c7d2-48af-b591-2975de15964c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:19.398931 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53188f36-c7d2-48af-b591-2975de15964c admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:19.460095 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333bbff-a154-4f40-ad31-2a657408e9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:19.460971 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333bbff-a154-4f40-ad31-2a657408e9d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:19.461254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333bbff-a154-4f40-ad31-2a657408e9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:19.461528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333bbff-a154-4f40-ad31-2a657408e9d5 admin admin] PATH_INFO: `/v3/projects/ec7426bd444d411a916e2b836c9e60e6` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:19.462751 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1333bbff-a154-4f40-ad31-2a657408e9d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec7426bd444d411a916e2b836c9e60e6)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:19.464673 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1333bbff-a154-4f40-ad31-2a657408e9d5 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:19.514945 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1333bbff-a154-4f40-ad31-2a657408e9d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec7426bd444d411a916e2b836c9e60e6', 'request_id': 'req-1333bbff-a154-4f40-ad31-2a657408e9d5'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:19.524892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1333bbff-a154-4f40-ad31-2a657408e9d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec7426bd444d411a916e2b836c9e60e6', 'request_id': 'req-1333bbff-a154-4f40-ad31-2a657408e9d5'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:19.532122 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1333bbff-a154-4f40-ad31-2a657408e9d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec7426bd444d411a916e2b836c9e60e6', 'request_id': 'req-1333bbff-a154-4f40-ad31-2a657408e9d5'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:19.545393 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1333bbff-a154-4f40-ad31-2a657408e9d5 admin admin] The token cache is being invalidate because project ec7426bd444d411a916e2b836c9e60e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:45:19.545679 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1333bbff-a154-4f40-ad31-2a657408e9d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:19.546410 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1333bbff-a154-4f40-ad31-2a657408e9d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:19.547967 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1316/2638] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:45:19 2022] DELETE /identity/v3/projects/ec7426bd444d411a916e2b836c9e60e6 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:45:19.551930 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e2ba0c-ef96-45fe-b094-68c1f2c7c1b7 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:19.553368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-b0e2ba0c-ef96-45fe-b094-68c1f2c7c1b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:19.566773 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e2ba0c-ef96-45fe-b094-68c1f2c7c1b7 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:19.625405 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025b40f-6182-497b-99e1-6d5263d2b510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:19.626230 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025b40f-6182-497b-99e1-6d5263d2b510 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:19.626518 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025b40f-6182-497b-99e1-6d5263d2b510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:19.626796 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025b40f-6182-497b-99e1-6d5263d2b510 admin admin] PATH_INFO: `/v3/users/12c056ee9d0b4d58bfb7af1bb147a3c9` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:19.628001 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b025b40f-6182-497b-99e1-6d5263d2b510 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12c056ee9d0b4d58bfb7af1bb147a3c9)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:19.629939 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b025b40f-6182-497b-99e1-6d5263d2b510 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:19.682027 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-b025b40f-6182-497b-99e1-6d5263d2b510 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12c056ee9d0b4d58bfb7af1bb147a3c9'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:19.683019 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-b025b40f-6182-497b-99e1-6d5263d2b510 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12c056ee9d0b4d58bfb7af1bb147a3c9'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:19.692157 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-b025b40f-6182-497b-99e1-6d5263d2b510 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12c056ee9d0b4d58bfb7af1bb147a3c9'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:19.692865 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-b025b40f-6182-497b-99e1-6d5263d2b510 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12c056ee9d0b4d58bfb7af1bb147a3c9'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:19.694989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:19.701420 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-b025b40f-6182-497b-99e1-6d5263d2b510 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12c056ee9d0b4d58bfb7af1bb147a3c9'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:19.720780 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1323/2639] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:45:19 2022] DELETE /identity/v3/users/12c056ee9d0b4d58bfb7af1bb147a3c9 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:45:20.009323 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333bbff-a154-4f40-ad31-2a657408e9d5 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:20.013559 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1333bbff-a154-4f40-ad31-2a657408e9d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:20.026593 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333bbff-a154-4f40-ad31-2a657408e9d5 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:20.091616 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a067e1f-40df-420d-8a69-26a310e8ef7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:20.092481 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a067e1f-40df-420d-8a69-26a310e8ef7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:20.092765 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a067e1f-40df-420d-8a69-26a310e8ef7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:20.093041 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a067e1f-40df-420d-8a69-26a310e8ef7b admin admin] PATH_INFO: `/v3/projects/0ddae7b20db14afba00a91dffdeb761f` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:20.094269 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a067e1f-40df-420d-8a69-26a310e8ef7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ddae7b20db14afba00a91dffdeb761f)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:20.096186 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a067e1f-40df-420d-8a69-26a310e8ef7b admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:20.133610 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:20.150282 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5a067e1f-40df-420d-8a69-26a310e8ef7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ddae7b20db14afba00a91dffdeb761f', 'request_id': 'req-5a067e1f-40df-420d-8a69-26a310e8ef7b'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:20.158382 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5a067e1f-40df-420d-8a69-26a310e8ef7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ddae7b20db14afba00a91dffdeb761f', 'request_id': 'req-5a067e1f-40df-420d-8a69-26a310e8ef7b'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:20.166711 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5a067e1f-40df-420d-8a69-26a310e8ef7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ddae7b20db14afba00a91dffdeb761f', 'request_id': 'req-5a067e1f-40df-420d-8a69-26a310e8ef7b'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:20.177741 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5a067e1f-40df-420d-8a69-26a310e8ef7b admin admin] The token cache is being invalidate because project 0ddae7b20db14afba00a91dffdeb761f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:45:20.178033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5a067e1f-40df-420d-8a69-26a310e8ef7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:20.178742 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5a067e1f-40df-420d-8a69-26a310e8ef7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:20.180746 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1317/2640] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:45:20 2022] DELETE /identity/v3/projects/0ddae7b20db14afba00a91dffdeb761f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:45:20.187478 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63c3bd-c710-42fe-adb9-60514d3b6601 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:20.187784 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63c3bd-c710-42fe-adb9-60514d3b6601 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:20.188402 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63c3bd-c710-42fe-adb9-60514d3b6601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:20.215566 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-5e63c3bd-c710-42fe-adb9-60514d3b6601 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:45:20.217405 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-5e63c3bd-c710-42fe-adb9-60514d3b6601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:20.293891 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1324/2641] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:45:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:45:20.302170 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55330d0-db55-489f-bd2c-18583ca32f1f None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:20.302508 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55330d0-db55-489f-bd2c-18583ca32f1f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:20.302823 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55330d0-db55-489f-bd2c-18583ca32f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:20.322049 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-a55330d0-db55-489f-bd2c-18583ca32f1f None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:45:20.323311 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a55330d0-db55-489f-bd2c-18583ca32f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:20.332864 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1318/2642] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:45:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:45:20.337495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63c3bd-c710-42fe-adb9-60514d3b6601 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:20.348799 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63c3bd-c710-42fe-adb9-60514d3b6601 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:20.354351 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970825d9-97c0-4b20-9638-3295bd9a4d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:20.355086 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970825d9-97c0-4b20-9638-3295bd9a4d85 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:20.355485 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970825d9-97c0-4b20-9638-3295bd9a4d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:20.355826 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970825d9-97c0-4b20-9638-3295bd9a4d85 admin admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:20.356795 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970825d9-97c0-4b20-9638-3295bd9a4d85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:45:20.357141 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970825d9-97c0-4b20-9638-3295bd9a4d85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:20.359082 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970825d9-97c0-4b20-9638-3295bd9a4d85 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:20.375744 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1325/2643] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:45:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:45:20.380438 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55330d0-db55-489f-bd2c-18583ca32f1f None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:20.391273 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55330d0-db55-489f-bd2c-18583ca32f1f None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:20.396339 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44eeea6-3778-4a3c-9f79-f3dec17e1d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:20.397050 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44eeea6-3778-4a3c-9f79-f3dec17e1d66 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:20.397356 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44eeea6-3778-4a3c-9f79-f3dec17e1d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:20.397623 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44eeea6-3778-4a3c-9f79-f3dec17e1d66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:20.398360 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44eeea6-3778-4a3c-9f79-f3dec17e1d66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:20.400400 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44eeea6-3778-4a3c-9f79-f3dec17e1d66 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:20.434126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1319/2644] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:45:20 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:45:20.437818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970825d9-97c0-4b20-9638-3295bd9a4d85 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:20.448126 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970825d9-97c0-4b20-9638-3295bd9a4d85 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:20.507187 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc146c-3caf-4e96-89d5-bba556eec726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:20.508017 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc146c-3caf-4e96-89d5-bba556eec726 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:20.508310 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc146c-3caf-4e96-89d5-bba556eec726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:20.508618 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc146c-3caf-4e96-89d5-bba556eec726 admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:20.509323 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc146c-3caf-4e96-89d5-bba556eec726 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:20.511293 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc146c-3caf-4e96-89d5-bba556eec726 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:20.532726 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1326/2645] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:45:20 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:45:20.536873 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44eeea6-3778-4a3c-9f79-f3dec17e1d66 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:20.547877 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44eeea6-3778-4a3c-9f79-f3dec17e1d66 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:20.552525 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b1cdf-e27e-478c-8272-b0895df1d09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:20.553196 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b1cdf-e27e-478c-8272-b0895df1d09b admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:20.553476 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b1cdf-e27e-478c-8272-b0895df1d09b admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:20.553750 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b1cdf-e27e-478c-8272-b0895df1d09b admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:20.554521 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877b1cdf-e27e-478c-8272-b0895df1d09b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:20.556118 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877b1cdf-e27e-478c-8272-b0895df1d09b admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:20.568148 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1320/2646] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:45:20.572559 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc146c-3caf-4e96-89d5-bba556eec726 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:20.585364 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc146c-3caf-4e96-89d5-bba556eec726 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:20.590035 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaa2936-1c0e-48a8-a492-cb255bd695f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:20.590810 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa2936-1c0e-48a8-a492-cb255bd695f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:20.591100 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa2936-1c0e-48a8-a492-cb255bd695f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:20.591382 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa2936-1c0e-48a8-a492-cb255bd695f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:20.592322 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaa2936-1c0e-48a8-a492-cb255bd695f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:20.594024 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaa2936-1c0e-48a8-a492-cb255bd695f4 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:20.604561 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1327/2647] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:45:20.609282 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b1cdf-e27e-478c-8272-b0895df1d09b admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:20.620429 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b1cdf-e27e-478c-8272-b0895df1d09b admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:20.625620 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7770000-885e-41db-b36a-9950e13cf774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:20.626548 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7770000-885e-41db-b36a-9950e13cf774 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:20.626832 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7770000-885e-41db-b36a-9950e13cf774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:20.627113 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7770000-885e-41db-b36a-9950e13cf774 admin admin] PATH_INFO: `/v3/projects/aafeedc33991462da96f4c238fe14256/users/efadf6027540453a975d3dadb3f676c5/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:20.644149 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7770000-885e-41db-b36a-9950e13cf774 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aafeedc33991462da96f4c238fe14256, user_id=efadf6027540453a975d3dadb3f676c5, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:20.646786 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7770000-885e-41db-b36a-9950e13cf774 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:20.660709 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1321/2648] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:45:20 2022] PUT /identity/v3/projects/aafeedc33991462da96f4c238fe14256/users/efadf6027540453a975d3dadb3f676c5/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:45:20.666061 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafc829-37b7-4edf-8623-a8da6ef758f7 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:20.666451 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafc829-37b7-4edf-8623-a8da6ef758f7 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:20.666966 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafc829-37b7-4edf-8623-a8da6ef758f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:20.685626 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-fdafc829-37b7-4edf-8623-a8da6ef758f7 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:45:20.686906 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-fdafc829-37b7-4edf-8623-a8da6ef758f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:20.722837 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:20.750996 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1328/2649] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:45:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:45:20.760654 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7770000-885e-41db-b36a-9950e13cf774 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:20.763007 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e7770000-885e-41db-b36a-9950e13cf774 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:20.818220 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7770000-885e-41db-b36a-9950e13cf774 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:20.822099 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6949c9-d0a9-4561-98aa-d7b8d0a82c85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:20.822833 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6949c9-d0a9-4561-98aa-d7b8d0a82c85 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:20.823195 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6949c9-d0a9-4561-98aa-d7b8d0a82c85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:20.823468 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6949c9-d0a9-4561-98aa-d7b8d0a82c85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:20.834658 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6949c9-d0a9-4561-98aa-d7b8d0a82c85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:20.836118 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6949c9-d0a9-4561-98aa-d7b8d0a82c85 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:20.846566 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1322/2650] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:45:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:45:21.490701 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f4fbd-ab7e-47f4-9b71-10f325b25dca None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:21.491141 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f4fbd-ab7e-47f4-9b71-10f325b25dca None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:21.491372 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f4fbd-ab7e-47f4-9b71-10f325b25dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:21.520942 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-455f4fbd-ab7e-47f4-9b71-10f325b25dca None None] MFA Rules not processed for user `efadf6027540453a975d3dadb3f676c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:45:21.522314 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-455f4fbd-ab7e-47f4-9b71-10f325b25dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:21.593378 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1329/2651] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:45:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:45:21.604365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6949c9-d0a9-4561-98aa-d7b8d0a82c85 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:21.605853 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1a6949c9-d0a9-4561-98aa-d7b8d0a82c85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:21.668355 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6949c9-d0a9-4561-98aa-d7b8d0a82c85 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:21.672571 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c9e06f-bf8c-446c-9cd9-80f8b53b4ebe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:21.675731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9e06f-bf8c-446c-9cd9-80f8b53b4ebe service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:21.675731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9e06f-bf8c-446c-9cd9-80f8b53b4ebe service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:21.675731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9e06f-bf8c-446c-9cd9-80f8b53b4ebe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:21.683434 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c9e06f-bf8c-446c-9cd9-80f8b53b4ebe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:21.685496 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c9e06f-bf8c-446c-9cd9-80f8b53b4ebe service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:21.690160 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:21.696377 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1323/2652] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:45:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:45:24.119717 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec17be-7ae7-4c52-98ab-de37e436be51 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:24.120184 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec17be-7ae7-4c52-98ab-de37e436be51 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:24.120300 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec17be-7ae7-4c52-98ab-de37e436be51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:24.140131 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-60ec17be-7ae7-4c52-98ab-de37e436be51 None None] MFA Rules not processed for user `efadf6027540453a975d3dadb3f676c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:45:24.141397 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-60ec17be-7ae7-4c52-98ab-de37e436be51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:24.148860 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1330/2653] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:45:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:45:34.133673 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=44 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:46.919695 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4d472-8aab-46af-b034-6d305fc8d740 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:46.919695 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4d472-8aab-46af-b034-6d305fc8d740 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:46.919695 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4d472-8aab-46af-b034-6d305fc8d740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:46.938588 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-abc4d472-8aab-46af-b034-6d305fc8d740 None None] MFA Rules not processed for user `efadf6027540453a975d3dadb3f676c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:45:46.940038 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-abc4d472-8aab-46af-b034-6d305fc8d740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:46.948052 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1324/2654] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:45:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:45:56.933061 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:58.653091 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec17be-7ae7-4c52-98ab-de37e436be51 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:58.655475 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-60ec17be-7ae7-4c52-98ab-de37e436be51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:58.680237 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec17be-7ae7-4c52-98ab-de37e436be51 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:58.686682 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d44b2-1b0f-4dd8-b971-b55508ff4d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:58.687589 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d44b2-1b0f-4dd8-b971-b55508ff4d7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:58.687870 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d44b2-1b0f-4dd8-b971-b55508ff4d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:58.688144 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d44b2-1b0f-4dd8-b971-b55508ff4d7d admin admin] PATH_INFO: `/v3/users/652e58dff72d437ea9b90fd096ca765c` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:58.689486 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08d44b2-1b0f-4dd8-b971-b55508ff4d7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=652e58dff72d437ea9b90fd096ca765c)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:58.691507 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08d44b2-1b0f-4dd8-b971-b55508ff4d7d admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:58.763654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-c08d44b2-1b0f-4dd8-b971-b55508ff4d7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '652e58dff72d437ea9b90fd096ca765c'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:58.764381 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-c08d44b2-1b0f-4dd8-b971-b55508ff4d7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '652e58dff72d437ea9b90fd096ca765c'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:58.773788 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-c08d44b2-1b0f-4dd8-b971-b55508ff4d7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '652e58dff72d437ea9b90fd096ca765c'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:58.774545 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-c08d44b2-1b0f-4dd8-b971-b55508ff4d7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '652e58dff72d437ea9b90fd096ca765c'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:58.783029 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-c08d44b2-1b0f-4dd8-b971-b55508ff4d7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '652e58dff72d437ea9b90fd096ca765c'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:58.802103 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1331/2655] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:45:58 2022] DELETE /identity/v3/users/652e58dff72d437ea9b90fd096ca765c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:45:58.806611 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e10d0-d5b8-44aa-bde1-ed5ea6e005dd None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:58.807106 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e10d0-d5b8-44aa-bde1-ed5ea6e005dd None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:58.807295 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e10d0-d5b8-44aa-bde1-ed5ea6e005dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:58.826997 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-638e10d0-d5b8-44aa-bde1-ed5ea6e005dd None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:45:58.828212 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-638e10d0-d5b8-44aa-bde1-ed5ea6e005dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:58.905376 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1325/2656] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:45:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:45:58.916177 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d44b2-1b0f-4dd8-b971-b55508ff4d7d admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:58.918249 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-c08d44b2-1b0f-4dd8-b971-b55508ff4d7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:58.984933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d44b2-1b0f-4dd8-b971-b55508ff4d7d admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:58.988649 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff855c6f-7cf4-46e5-a67f-e9d1f45f5a7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:58.989311 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff855c6f-7cf4-46e5-a67f-e9d1f45f5a7e service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:58.989594 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff855c6f-7cf4-46e5-a67f-e9d1f45f5a7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:58.989869 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff855c6f-7cf4-46e5-a67f-e9d1f45f5a7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:59.001015 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff855c6f-7cf4-46e5-a67f-e9d1f45f5a7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:59.002519 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff855c6f-7cf4-46e5-a67f-e9d1f45f5a7e service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:59.013236 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1332/2657] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:45:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:45:59.352219 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e10d0-d5b8-44aa-bde1-ed5ea6e005dd None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:59.353913 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-638e10d0-d5b8-44aa-bde1-ed5ea6e005dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:59.373461 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e10d0-d5b8-44aa-bde1-ed5ea6e005dd None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:59.380789 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e167d-effb-4053-96f2-7fd8722c4798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:59.381850 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e167d-effb-4053-96f2-7fd8722c4798 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:59.382212 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e167d-effb-4053-96f2-7fd8722c4798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:59.382503 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e167d-effb-4053-96f2-7fd8722c4798 admin admin] PATH_INFO: `/v3/projects/f6b10aa1658d47d0b8217a7013d67950` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:59.384013 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320e167d-effb-4053-96f2-7fd8722c4798 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6b10aa1658d47d0b8217a7013d67950)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:59.386238 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320e167d-effb-4053-96f2-7fd8722c4798 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:59.461034 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-320e167d-effb-4053-96f2-7fd8722c4798 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6b10aa1658d47d0b8217a7013d67950', 'request_id': 'req-320e167d-effb-4053-96f2-7fd8722c4798'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:59.474417 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-320e167d-effb-4053-96f2-7fd8722c4798 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6b10aa1658d47d0b8217a7013d67950', 'request_id': 'req-320e167d-effb-4053-96f2-7fd8722c4798'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:59.494130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-320e167d-effb-4053-96f2-7fd8722c4798 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6b10aa1658d47d0b8217a7013d67950', 'request_id': 'req-320e167d-effb-4053-96f2-7fd8722c4798'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:59.506078 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-320e167d-effb-4053-96f2-7fd8722c4798 admin admin] The token cache is being invalidate because project f6b10aa1658d47d0b8217a7013d67950 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:45:59.507586 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-320e167d-effb-4053-96f2-7fd8722c4798 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:59.507586 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-320e167d-effb-4053-96f2-7fd8722c4798 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:59.513063 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1326/2658] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:45:59 2022] DELETE /identity/v3/projects/f6b10aa1658d47d0b8217a7013d67950 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:45:59.514648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58595f8c-2ade-419f-8d4a-e848f6336d3e None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:59.515170 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58595f8c-2ade-419f-8d4a-e848f6336d3e None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:59.515448 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58595f8c-2ade-419f-8d4a-e848f6336d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:59.549399 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-58595f8c-2ade-419f-8d4a-e848f6336d3e None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:45:59.551625 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-58595f8c-2ade-419f-8d4a-e848f6336d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:59.631488 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e167d-effb-4053-96f2-7fd8722c4798 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:59.633356 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-320e167d-effb-4053-96f2-7fd8722c4798 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:59.637211 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1333/2659] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:45:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:45:59.648977 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116dbf68-b2f8-43d2-a2bc-ef5ecf37a29c None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:59.649360 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116dbf68-b2f8-43d2-a2bc-ef5ecf37a29c None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:59.650162 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116dbf68-b2f8-43d2-a2bc-ef5ecf37a29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:59.654654 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e167d-effb-4053-96f2-7fd8722c4798 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:59.667238 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b9a24-0d26-4712-95db-3c2f9fbcd6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:59.668149 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b9a24-0d26-4712-95db-3c2f9fbcd6c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:59.668431 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b9a24-0d26-4712-95db-3c2f9fbcd6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:59.668918 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b9a24-0d26-4712-95db-3c2f9fbcd6c5 admin admin] PATH_INFO: `/v3/users/8f93a3d32f6c4151be0d7493c693c377` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:59.670526 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04b9a24-0d26-4712-95db-3c2f9fbcd6c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f93a3d32f6c4151be0d7493c693c377)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:59.670973 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-116dbf68-b2f8-43d2-a2bc-ef5ecf37a29c None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:45:59.672219 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-116dbf68-b2f8-43d2-a2bc-ef5ecf37a29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:59.672717 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04b9a24-0d26-4712-95db-3c2f9fbcd6c5 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:59.683490 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1334/2660] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:45:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:45:59.691897 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116dbf68-b2f8-43d2-a2bc-ef5ecf37a29c None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:59.703117 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116dbf68-b2f8-43d2-a2bc-ef5ecf37a29c None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:59.708872 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefea68d-9bc2-485b-96cb-18f81050f663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:59.709720 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefea68d-9bc2-485b-96cb-18f81050f663 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:59.710415 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefea68d-9bc2-485b-96cb-18f81050f663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:59.710865 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefea68d-9bc2-485b-96cb-18f81050f663 admin admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:59.712123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefea68d-9bc2-485b-96cb-18f81050f663 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:45:59.712723 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefea68d-9bc2-485b-96cb-18f81050f663 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:59.715371 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefea68d-9bc2-485b-96cb-18f81050f663 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:59.719477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:59.741374 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1335/2661] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:45:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:45:59.741701 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefea68d-9bc2-485b-96cb-18f81050f663 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:59.754230 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefea68d-9bc2-485b-96cb-18f81050f663 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:59.757008 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f04b9a24-0d26-4712-95db-3c2f9fbcd6c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f93a3d32f6c4151be0d7493c693c377'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:59.758067 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f04b9a24-0d26-4712-95db-3c2f9fbcd6c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f93a3d32f6c4151be0d7493c693c377'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:59.759311 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f04b9a24-0d26-4712-95db-3c2f9fbcd6c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f93a3d32f6c4151be0d7493c693c377'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:59.761044 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956c991-7a2e-4f83-97c9-f7d0f694565c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:59.762183 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956c991-7a2e-4f83-97c9-f7d0f694565c admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:59.762933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956c991-7a2e-4f83-97c9-f7d0f694565c admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:59.763397 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956c991-7a2e-4f83-97c9-f7d0f694565c admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:59.764570 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c956c991-7a2e-4f83-97c9-f7d0f694565c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:59.766558 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c956c991-7a2e-4f83-97c9-f7d0f694565c admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:45:59.784753 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f04b9a24-0d26-4712-95db-3c2f9fbcd6c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f93a3d32f6c4151be0d7493c693c377'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:59.799614 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f04b9a24-0d26-4712-95db-3c2f9fbcd6c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f93a3d32f6c4151be0d7493c693c377'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:45:59.800393 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1336/2662] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:45:59 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:45:59.805683 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:45:59.806756 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956c991-7a2e-4f83-97c9-f7d0f694565c admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:59.810070 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-c956c991-7a2e-4f83-97c9-f7d0f694565c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:59.821398 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1327/2663] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:45:59 2022] DELETE /identity/v3/users/8f93a3d32f6c4151be0d7493c693c377 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:45:59.828153 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf47977-71c2-423d-99a6-306eaf4673e2 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:59.828623 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf47977-71c2-423d-99a6-306eaf4673e2 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:59.829074 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf47977-71c2-423d-99a6-306eaf4673e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:59.858803 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-faf47977-71c2-423d-99a6-306eaf4673e2 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:45:59.860863 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-faf47977-71c2-423d-99a6-306eaf4673e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:59.885929 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956c991-7a2e-4f83-97c9-f7d0f694565c admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:45:59.957947 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1328/2664] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:45:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:45:59.969607 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf47977-71c2-423d-99a6-306eaf4673e2 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:45:59.970329 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b3f52-2441-4a26-a17c-5bdbf695ec91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:45:59.971253 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-faf47977-71c2-423d-99a6-306eaf4673e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:45:59.971553 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b3f52-2441-4a26-a17c-5bdbf695ec91 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:45:59.975296 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b3f52-2441-4a26-a17c-5bdbf695ec91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:45:59.975296 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b3f52-2441-4a26-a17c-5bdbf695ec91 admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:45:59.976712 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7b3f52-2441-4a26-a17c-5bdbf695ec91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:45:59.977989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7b3f52-2441-4a26-a17c-5bdbf695ec91 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:00.001300 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1337/2665] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:45:59 2022] POST /identity/v3/users => generated 286 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:46:00.005515 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b3f52-2441-4a26-a17c-5bdbf695ec91 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:00.018081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b3f52-2441-4a26-a17c-5bdbf695ec91 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:00.032891 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d3992-21de-431b-9ab6-d175498600cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:00.034112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d3992-21de-431b-9ab6-d175498600cc admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:00.034963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d3992-21de-431b-9ab6-d175498600cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:00.035455 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d3992-21de-431b-9ab6-d175498600cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:00.036802 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4d3992-21de-431b-9ab6-d175498600cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:00.039850 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4d3992-21de-431b-9ab6-d175498600cc admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:00.051963 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf47977-71c2-423d-99a6-306eaf4673e2 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:00.056263 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c72de4d-8dbc-4b6c-b49e-e71e0d69a661 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:00.056927 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c72de4d-8dbc-4b6c-b49e-e71e0d69a661 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:00.057811 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c72de4d-8dbc-4b6c-b49e-e71e0d69a661 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:00.058109 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c72de4d-8dbc-4b6c-b49e-e71e0d69a661 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:00.060640 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1338/2666] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:46:00.065877 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d3992-21de-431b-9ab6-d175498600cc admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:00.068866 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c72de4d-8dbc-4b6c-b49e-e71e0d69a661 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:00.070398 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c72de4d-8dbc-4b6c-b49e-e71e0d69a661 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:00.082601 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d3992-21de-431b-9ab6-d175498600cc admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:00.084909 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1329/2667] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:45:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:46:00.089374 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df23d612-06ff-4b2d-a463-fd08cd9e3e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:00.090075 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23d612-06ff-4b2d-a463-fd08cd9e3e55 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:00.090407 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23d612-06ff-4b2d-a463-fd08cd9e3e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:00.090687 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23d612-06ff-4b2d-a463-fd08cd9e3e55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:00.091456 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df23d612-06ff-4b2d-a463-fd08cd9e3e55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:00.094256 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df23d612-06ff-4b2d-a463-fd08cd9e3e55 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:00.111646 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1339/2668] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:46:00.117546 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c72de4d-8dbc-4b6c-b49e-e71e0d69a661 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:00.139009 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c72de4d-8dbc-4b6c-b49e-e71e0d69a661 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:00.146808 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372ceab1-41f3-49ef-b0ad-f72d41b6405d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:00.148608 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ceab1-41f3-49ef-b0ad-f72d41b6405d admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:00.149598 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ceab1-41f3-49ef-b0ad-f72d41b6405d admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:00.150317 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ceab1-41f3-49ef-b0ad-f72d41b6405d admin admin] PATH_INFO: `/v3/projects/7702ad43c7b74511ba831f857dd885ec/users/03f7655b728246e0b9973f838ed61fac/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:00.171065 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372ceab1-41f3-49ef-b0ad-f72d41b6405d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7702ad43c7b74511ba831f857dd885ec, user_id=03f7655b728246e0b9973f838ed61fac, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:00.173927 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372ceab1-41f3-49ef-b0ad-f72d41b6405d admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:00.194501 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1330/2669] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:46:00 2022] PUT /identity/v3/projects/7702ad43c7b74511ba831f857dd885ec/users/03f7655b728246e0b9973f838ed61fac/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:00.205268 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ed49b-1839-4c30-bdf6-1604bcea5d81 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:00.205551 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ed49b-1839-4c30-bdf6-1604bcea5d81 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:00.205830 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ed49b-1839-4c30-bdf6-1604bcea5d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:00.244139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-580ed49b-1839-4c30-bdf6-1604bcea5d81 None None] MFA Rules not processed for user `03f7655b728246e0b9973f838ed61fac`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:00.246605 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-580ed49b-1839-4c30-bdf6-1604bcea5d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:00.302189 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:00.359865 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1340/2670] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:46:00.375222 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372ceab1-41f3-49ef-b0ad-f72d41b6405d admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:00.378046 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-372ceab1-41f3-49ef-b0ad-f72d41b6405d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:00.469224 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ed49b-1839-4c30-bdf6-1604bcea5d81 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:00.470865 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-580ed49b-1839-4c30-bdf6-1604bcea5d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:00.476578 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372ceab1-41f3-49ef-b0ad-f72d41b6405d admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:00.481113 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb13cb94-4808-4d8b-9359-37a9e5da052d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:00.482149 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13cb94-4808-4d8b-9359-37a9e5da052d service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:00.482603 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13cb94-4808-4d8b-9359-37a9e5da052d service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:00.483057 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13cb94-4808-4d8b-9359-37a9e5da052d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:00.493238 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb13cb94-4808-4d8b-9359-37a9e5da052d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:00.494658 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb13cb94-4808-4d8b-9359-37a9e5da052d service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:00.507695 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1331/2671] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:46:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:46:00.558161 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb13cb94-4808-4d8b-9359-37a9e5da052d service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:00.601894 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:00.626174 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ed49b-1839-4c30-bdf6-1604bcea5d81 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:00.631183 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275fe1e-0e90-4d4f-95fb-647c6ac06393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:00.632074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275fe1e-0e90-4d4f-95fb-647c6ac06393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:00.632363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275fe1e-0e90-4d4f-95fb-647c6ac06393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:00.632648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275fe1e-0e90-4d4f-95fb-647c6ac06393 admin admin] PATH_INFO: `/v3/projects/203972aef723487d950c600501303a94` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:00.633929 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9275fe1e-0e90-4d4f-95fb-647c6ac06393 admin admin] RBAC: Authorizing `identity:delete_project(project_id=203972aef723487d950c600501303a94)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:00.636492 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9275fe1e-0e90-4d4f-95fb-647c6ac06393 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:00.666583 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb13cb94-4808-4d8b-9359-37a9e5da052d service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:00.673297 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da98031-efa7-4c1e-8361-905ede841259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:00.674479 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da98031-efa7-4c1e-8361-905ede841259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:00.674981 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da98031-efa7-4c1e-8361-905ede841259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:00.675480 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da98031-efa7-4c1e-8361-905ede841259 admin admin] PATH_INFO: `/v3/users/03f7655b728246e0b9973f838ed61fac` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:00.677621 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da98031-efa7-4c1e-8361-905ede841259 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03f7655b728246e0b9973f838ed61fac)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:00.680125 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da98031-efa7-4c1e-8361-905ede841259 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:00.694191 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-9275fe1e-0e90-4d4f-95fb-647c6ac06393 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '203972aef723487d950c600501303a94', 'request_id': 'req-9275fe1e-0e90-4d4f-95fb-647c6ac06393'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:00.705037 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-9275fe1e-0e90-4d4f-95fb-647c6ac06393 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '203972aef723487d950c600501303a94', 'request_id': 'req-9275fe1e-0e90-4d4f-95fb-647c6ac06393'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:00.713003 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-9275fe1e-0e90-4d4f-95fb-647c6ac06393 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '203972aef723487d950c600501303a94', 'request_id': 'req-9275fe1e-0e90-4d4f-95fb-647c6ac06393'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:00.730108 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-9275fe1e-0e90-4d4f-95fb-647c6ac06393 admin admin] The token cache is being invalidate because project 203972aef723487d950c600501303a94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:46:00.730440 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-9275fe1e-0e90-4d4f-95fb-647c6ac06393 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:00.731131 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-9275fe1e-0e90-4d4f-95fb-647c6ac06393 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:00.733251 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1341/2672] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:46:00 2022] DELETE /identity/v3/projects/203972aef723487d950c600501303a94 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:00.741882 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275fe1e-0e90-4d4f-95fb-647c6ac06393 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:00.743478 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-9275fe1e-0e90-4d4f-95fb-647c6ac06393 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:00.761910 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2da98031-efa7-4c1e-8361-905ede841259 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03f7655b728246e0b9973f838ed61fac'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:00.762969 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2da98031-efa7-4c1e-8361-905ede841259 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03f7655b728246e0b9973f838ed61fac'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:00.763932 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2da98031-efa7-4c1e-8361-905ede841259 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03f7655b728246e0b9973f838ed61fac'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:00.785294 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2da98031-efa7-4c1e-8361-905ede841259 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03f7655b728246e0b9973f838ed61fac'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:00.792896 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2da98031-efa7-4c1e-8361-905ede841259 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03f7655b728246e0b9973f838ed61fac'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:00.808293 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1332/2673] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:46:00 2022] DELETE /identity/v3/users/03f7655b728246e0b9973f838ed61fac => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:46:00.813009 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f4645-3b2c-469b-bf0e-cbfca3c509f5 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:00.813292 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f4645-3b2c-469b-bf0e-cbfca3c509f5 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:00.813580 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f4645-3b2c-469b-bf0e-cbfca3c509f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:00.825403 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275fe1e-0e90-4d4f-95fb-647c6ac06393 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:00.837088 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-5f2f4645-3b2c-469b-bf0e-cbfca3c509f5 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:00.838379 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-5f2f4645-3b2c-469b-bf0e-cbfca3c509f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:00.841422 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:00.904115 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a97d2-c2a1-4625-b88c-8fc839a3cb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:00.905062 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a97d2-c2a1-4625-b88c-8fc839a3cb0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:00.905311 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a97d2-c2a1-4625-b88c-8fc839a3cb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:00.905591 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a97d2-c2a1-4625-b88c-8fc839a3cb0a admin admin] PATH_INFO: `/v3/users/2e4f4e49453c44cf907a654590803362` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:00.906922 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73a97d2-c2a1-4625-b88c-8fc839a3cb0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e4f4e49453c44cf907a654590803362)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:00.908867 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73a97d2-c2a1-4625-b88c-8fc839a3cb0a admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:00.920301 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1333/2674] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:46:00.931952 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f4645-3b2c-469b-bf0e-cbfca3c509f5 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:00.934262 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-5f2f4645-3b2c-469b-bf0e-cbfca3c509f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:00.973458 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-d73a97d2-c2a1-4625-b88c-8fc839a3cb0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e4f4e49453c44cf907a654590803362'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:00.974081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-d73a97d2-c2a1-4625-b88c-8fc839a3cb0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e4f4e49453c44cf907a654590803362'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:00.983973 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-d73a97d2-c2a1-4625-b88c-8fc839a3cb0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e4f4e49453c44cf907a654590803362'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:00.984929 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-d73a97d2-c2a1-4625-b88c-8fc839a3cb0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e4f4e49453c44cf907a654590803362'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:00.986484 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:00.995345 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-d73a97d2-c2a1-4625-b88c-8fc839a3cb0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e4f4e49453c44cf907a654590803362'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:01.009826 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f4645-3b2c-469b-bf0e-cbfca3c509f5 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:01.014606 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92afe7e-f42c-4802-a124-177108ba7189 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:01.015352 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92afe7e-f42c-4802-a124-177108ba7189 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:01.015735 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92afe7e-f42c-4802-a124-177108ba7189 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:01.016026 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92afe7e-f42c-4802-a124-177108ba7189 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:01.017973 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e92afe7e-f42c-4802-a124-177108ba7189 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:01.023211 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1342/2675] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:46:00 2022] DELETE /identity/v3/users/2e4f4e49453c44cf907a654590803362 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:01.033317 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92afe7e-f42c-4802-a124-177108ba7189 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:01.034849 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92afe7e-f42c-4802-a124-177108ba7189 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:01.046909 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1334/2676] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:46:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:46:01.451887 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a97d2-c2a1-4625-b88c-8fc839a3cb0a admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:01.453402 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-d73a97d2-c2a1-4625-b88c-8fc839a3cb0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:01.487161 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92afe7e-f42c-4802-a124-177108ba7189 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:01.488726 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e92afe7e-f42c-4802-a124-177108ba7189 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:01.502761 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92afe7e-f42c-4802-a124-177108ba7189 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:01.539409 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a97d2-c2a1-4625-b88c-8fc839a3cb0a admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:01.544450 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24488830-bde8-43b6-84b5-873200ae92fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:01.545348 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24488830-bde8-43b6-84b5-873200ae92fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:01.545623 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24488830-bde8-43b6-84b5-873200ae92fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:01.545899 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24488830-bde8-43b6-84b5-873200ae92fc admin admin] PATH_INFO: `/v3/projects/9257d5d0e871496e804457058584953c` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:01.547228 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24488830-bde8-43b6-84b5-873200ae92fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9257d5d0e871496e804457058584953c)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:01.549180 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24488830-bde8-43b6-84b5-873200ae92fc admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:01.582195 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3ea70-6b32-4c34-acad-342f3b253ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:01.583644 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3ea70-6b32-4c34-acad-342f3b253ca7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:01.584022 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3ea70-6b32-4c34-acad-342f3b253ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:01.584334 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3ea70-6b32-4c34-acad-342f3b253ca7 admin admin] PATH_INFO: `/v3/projects/7702ad43c7b74511ba831f857dd885ec` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:01.586766 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3ea70-6b32-4c34-acad-342f3b253ca7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7702ad43c7b74511ba831f857dd885ec)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:01.588265 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3ea70-6b32-4c34-acad-342f3b253ca7 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:01.603093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:01.631554 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-24488830-bde8-43b6-84b5-873200ae92fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9257d5d0e871496e804457058584953c', 'request_id': 'req-24488830-bde8-43b6-84b5-873200ae92fc'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:01.646782 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-24488830-bde8-43b6-84b5-873200ae92fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9257d5d0e871496e804457058584953c', 'request_id': 'req-24488830-bde8-43b6-84b5-873200ae92fc'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:01.656269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-24488830-bde8-43b6-84b5-873200ae92fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9257d5d0e871496e804457058584953c', 'request_id': 'req-24488830-bde8-43b6-84b5-873200ae92fc'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:01.667409 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-41f3ea70-6b32-4c34-acad-342f3b253ca7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7702ad43c7b74511ba831f857dd885ec', 'request_id': 'req-41f3ea70-6b32-4c34-acad-342f3b253ca7'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:01.676362 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-24488830-bde8-43b6-84b5-873200ae92fc admin admin] The token cache is being invalidate because project 9257d5d0e871496e804457058584953c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:46:01.676362 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-24488830-bde8-43b6-84b5-873200ae92fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:01.676362 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-24488830-bde8-43b6-84b5-873200ae92fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:01.678456 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1343/2677] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:46:01 2022] DELETE /identity/v3/projects/9257d5d0e871496e804457058584953c => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:01.683969 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-41f3ea70-6b32-4c34-acad-342f3b253ca7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7702ad43c7b74511ba831f857dd885ec', 'request_id': 'req-41f3ea70-6b32-4c34-acad-342f3b253ca7'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:01.685965 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad85c94-f67b-4ace-82b2-9f17769ad862 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:01.686440 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad85c94-f67b-4ace-82b2-9f17769ad862 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:01.686950 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad85c94-f67b-4ace-82b2-9f17769ad862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:01.696010 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-41f3ea70-6b32-4c34-acad-342f3b253ca7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7702ad43c7b74511ba831f857dd885ec', 'request_id': 'req-41f3ea70-6b32-4c34-acad-342f3b253ca7'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:01.712116 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-4ad85c94-f67b-4ace-82b2-9f17769ad862 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:01.713459 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-4ad85c94-f67b-4ace-82b2-9f17769ad862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:01.715514 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-41f3ea70-6b32-4c34-acad-342f3b253ca7 admin admin] The token cache is being invalidate because project 7702ad43c7b74511ba831f857dd885ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:46:01.715977 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-41f3ea70-6b32-4c34-acad-342f3b253ca7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:01.729110 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-41f3ea70-6b32-4c34-acad-342f3b253ca7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:01.734948 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1335/2678] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:46:01 2022] DELETE /identity/v3/projects/7702ad43c7b74511ba831f857dd885ec => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:01.746635 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f15fb0a-8834-49dc-b073-1961f994959c None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:01.746994 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f15fb0a-8834-49dc-b073-1961f994959c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:01.747327 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f15fb0a-8834-49dc-b073-1961f994959c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:01.776381 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-0f15fb0a-8834-49dc-b073-1961f994959c None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:01.777629 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0f15fb0a-8834-49dc-b073-1961f994959c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:01.799164 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1344/2679] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:46:01.806118 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9071f-9fb2-4431-81dc-f7604a08eddf None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:01.806492 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9071f-9fb2-4431-81dc-f7604a08eddf None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:01.806842 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9071f-9fb2-4431-81dc-f7604a08eddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:01.832099 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-b8d9071f-9fb2-4431-81dc-f7604a08eddf None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:01.833724 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-b8d9071f-9fb2-4431-81dc-f7604a08eddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:01.844768 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1345/2680] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:46:01.848709 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d9071f-9fb2-4431-81dc-f7604a08eddf None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:01.868047 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d9071f-9fb2-4431-81dc-f7604a08eddf None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:01.868047 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49979c05-18a4-4c8f-9f00-bcbde3c9c0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:01.868047 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49979c05-18a4-4c8f-9f00-bcbde3c9c0fc admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:01.868047 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49979c05-18a4-4c8f-9f00-bcbde3c9c0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:01.868755 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49979c05-18a4-4c8f-9f00-bcbde3c9c0fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:01.868755 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49979c05-18a4-4c8f-9f00-bcbde3c9c0fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:46:01.868755 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49979c05-18a4-4c8f-9f00-bcbde3c9c0fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:01.870440 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49979c05-18a4-4c8f-9f00-bcbde3c9c0fc admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:01.877009 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1336/2681] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:46:01.879211 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f280c-6dee-44da-90a7-de20c7f6cf12 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:01.880177 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f280c-6dee-44da-90a7-de20c7f6cf12 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:01.881140 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f280c-6dee-44da-90a7-de20c7f6cf12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:01.887389 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1346/2682] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:46:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:46:01.888579 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49979c05-18a4-4c8f-9f00-bcbde3c9c0fc admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:01.898918 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49979c05-18a4-4c8f-9f00-bcbde3c9c0fc admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:01.902334 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-cb1f280c-6dee-44da-90a7-de20c7f6cf12 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:01.903569 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-cb1f280c-6dee-44da-90a7-de20c7f6cf12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:01.907866 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed25f7e8-6793-4ce8-8834-4220268c8107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:01.908537 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25f7e8-6793-4ce8-8834-4220268c8107 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:01.908825 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25f7e8-6793-4ce8-8834-4220268c8107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:01.909105 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25f7e8-6793-4ce8-8834-4220268c8107 admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:01.909809 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed25f7e8-6793-4ce8-8834-4220268c8107 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:01.911960 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1337/2683] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:46:01.912083 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed25f7e8-6793-4ce8-8834-4220268c8107 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:01.916218 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f280c-6dee-44da-90a7-de20c7f6cf12 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:01.925033 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:01.931835 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f280c-6dee-44da-90a7-de20c7f6cf12 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:01.940482 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e7ebd9-2c51-4b2e-bf20-d88464c12409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:01.941511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7ebd9-2c51-4b2e-bf20-d88464c12409 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:01.942003 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7ebd9-2c51-4b2e-bf20-d88464c12409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:01.942348 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7ebd9-2c51-4b2e-bf20-d88464c12409 admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:01.943394 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e7ebd9-2c51-4b2e-bf20-d88464c12409 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:46:01.944780 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e7ebd9-2c51-4b2e-bf20-d88464c12409 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:01.946756 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1347/2684] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:46:01 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:46:01.950450 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e7ebd9-2c51-4b2e-bf20-d88464c12409 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:01.953564 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed25f7e8-6793-4ce8-8834-4220268c8107 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:01.967545 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1338/2685] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:46:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:46:01.972342 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e7ebd9-2c51-4b2e-bf20-d88464c12409 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:01.985780 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e7ebd9-2c51-4b2e-bf20-d88464c12409 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:02.075117 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed25f7e8-6793-4ce8-8834-4220268c8107 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:02.087738 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527e4b9-a517-4a73-a9e0-2c4e910bbf99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:02.090166 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8d6ed-ee45-4740-b88b-a30fc98656a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:02.090708 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527e4b9-a517-4a73-a9e0-2c4e910bbf99 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:02.091168 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527e4b9-a517-4a73-a9e0-2c4e910bbf99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:02.091637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527e4b9-a517-4a73-a9e0-2c4e910bbf99 admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:02.091810 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8d6ed-ee45-4740-b88b-a30fc98656a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:02.092231 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8d6ed-ee45-4740-b88b-a30fc98656a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:02.092677 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8d6ed-ee45-4740-b88b-a30fc98656a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:02.092840 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5527e4b9-a517-4a73-a9e0-2c4e910bbf99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:02.095288 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef8d6ed-ee45-4740-b88b-a30fc98656a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:02.096490 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5527e4b9-a517-4a73-a9e0-2c4e910bbf99 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:02.099423 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef8d6ed-ee45-4740-b88b-a30fc98656a1 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:02.130033 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1348/2686] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:46:01 2022] POST /identity/v3/users => generated 287 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:46:02.137015 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527e4b9-a517-4a73-a9e0-2c4e910bbf99 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:02.153273 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1339/2687] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:46:01 2022] POST /identity/v3/projects => generated 375 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:46:02.155578 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527e4b9-a517-4a73-a9e0-2c4e910bbf99 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:02.158724 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8d6ed-ee45-4740-b88b-a30fc98656a1 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:02.172205 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:02.274605 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7826e40-af0e-433f-ae05-fed18af7b90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:02.276281 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7826e40-af0e-433f-ae05-fed18af7b90b admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:02.278305 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7826e40-af0e-433f-ae05-fed18af7b90b admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:02.279127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7826e40-af0e-433f-ae05-fed18af7b90b admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:02.280911 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7826e40-af0e-433f-ae05-fed18af7b90b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:02.285061 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7826e40-af0e-433f-ae05-fed18af7b90b admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:02.299059 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8d6ed-ee45-4740-b88b-a30fc98656a1 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:02.303965 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f4218-3d74-4969-ae5c-9281530b3c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:02.305273 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f4218-3d74-4969-ae5c-9281530b3c95 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:02.305573 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f4218-3d74-4969-ae5c-9281530b3c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:02.305865 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f4218-3d74-4969-ae5c-9281530b3c95 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:02.306596 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f4218-3d74-4969-ae5c-9281530b3c95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:02.307735 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1349/2688] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:46:02 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:46:02.309295 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f4218-3d74-4969-ae5c-9281530b3c95 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:02.312171 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7826e40-af0e-433f-ae05-fed18af7b90b admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:02.327447 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7826e40-af0e-433f-ae05-fed18af7b90b admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:02.347694 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1340/2689] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:46:02 2022] POST /identity/v3/users => generated 288 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:46:02.348178 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dccf1e-13c5-454f-9ef9-6b55c042d6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:02.348849 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dccf1e-13c5-454f-9ef9-6b55c042d6f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:02.349135 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dccf1e-13c5-454f-9ef9-6b55c042d6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:02.349413 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dccf1e-13c5-454f-9ef9-6b55c042d6f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:02.350186 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dccf1e-13c5-454f-9ef9-6b55c042d6f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:02.353432 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f4218-3d74-4969-ae5c-9281530b3c95 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:02.353783 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dccf1e-13c5-454f-9ef9-6b55c042d6f7 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:02.366007 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f4218-3d74-4969-ae5c-9281530b3c95 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:02.367197 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1350/2690] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:46:02 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:46:02.376503 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905b288-ef32-4c66-977f-91c91b1acf69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:02.377323 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905b288-ef32-4c66-977f-91c91b1acf69 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:02.377670 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905b288-ef32-4c66-977f-91c91b1acf69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:02.378012 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905b288-ef32-4c66-977f-91c91b1acf69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:02.378891 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905b288-ef32-4c66-977f-91c91b1acf69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:02.380612 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905b288-ef32-4c66-977f-91c91b1acf69 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:02.385546 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dccf1e-13c5-454f-9ef9-6b55c042d6f7 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:02.392787 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1341/2691] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:46:02 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:46:02.396935 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dccf1e-13c5-454f-9ef9-6b55c042d6f7 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:02.397219 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905b288-ef32-4c66-977f-91c91b1acf69 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:02.405389 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18de4a88-2a30-4df1-a828-203e7fe33d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:02.406582 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de4a88-2a30-4df1-a828-203e7fe33d81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:02.406939 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de4a88-2a30-4df1-a828-203e7fe33d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:02.407292 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de4a88-2a30-4df1-a828-203e7fe33d81 admin admin] PATH_INFO: `/v3/projects/e0b3cf123a284b7ba783dea56e84581e/users/944db5934ce2464992caf38fabd85db6/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:02.411973 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905b288-ef32-4c66-977f-91c91b1acf69 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:02.418426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc8a42-eb75-41fc-98ad-650eeed9a816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:02.419699 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc8a42-eb75-41fc-98ad-650eeed9a816 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:02.420149 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc8a42-eb75-41fc-98ad-650eeed9a816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:02.420792 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc8a42-eb75-41fc-98ad-650eeed9a816 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:02.421753 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cc8a42-eb75-41fc-98ad-650eeed9a816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:02.423656 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cc8a42-eb75-41fc-98ad-650eeed9a816 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:02.426142 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18de4a88-2a30-4df1-a828-203e7fe33d81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0b3cf123a284b7ba783dea56e84581e, user_id=944db5934ce2464992caf38fabd85db6, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:02.429010 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18de4a88-2a30-4df1-a828-203e7fe33d81 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:02.443338 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1342/2692] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:46:02 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:46:02.447811 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc8a42-eb75-41fc-98ad-650eeed9a816 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:02.468423 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1351/2693] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:46:02 2022] PUT /identity/v3/projects/e0b3cf123a284b7ba783dea56e84581e/users/944db5934ce2464992caf38fabd85db6/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:46:02.469569 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc8a42-eb75-41fc-98ad-650eeed9a816 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:02.471873 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47dcd6-098c-44f1-8fef-e45eb420a9c2 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:02.472329 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47dcd6-098c-44f1-8fef-e45eb420a9c2 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:02.472765 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47dcd6-098c-44f1-8fef-e45eb420a9c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:02.522494 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-eb47dcd6-098c-44f1-8fef-e45eb420a9c2 None None] MFA Rules not processed for user `944db5934ce2464992caf38fabd85db6`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:02.524093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-eb47dcd6-098c-44f1-8fef-e45eb420a9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:02.543050 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:02.564869 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ee7dd-4c49-4be1-93bb-a838149ab2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:02.566020 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ee7dd-4c49-4be1-93bb-a838149ab2b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:02.567098 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ee7dd-4c49-4be1-93bb-a838149ab2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:02.567498 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ee7dd-4c49-4be1-93bb-a838149ab2b3 admin admin] PATH_INFO: `/v3/projects/64a686a0b5d04802981b602dceb98804/users/b65a1f14f9bc46c59214ed240316aa86/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:02.594275 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34ee7dd-4c49-4be1-93bb-a838149ab2b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64a686a0b5d04802981b602dceb98804, user_id=b65a1f14f9bc46c59214ed240316aa86, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:02.597023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34ee7dd-4c49-4be1-93bb-a838149ab2b3 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:02.633554 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1352/2694] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:46:02.633950 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1343/2695] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:46:02 2022] PUT /identity/v3/projects/64a686a0b5d04802981b602dceb98804/users/b65a1f14f9bc46c59214ed240316aa86/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:46:02.635647 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb47dcd6-098c-44f1-8fef-e45eb420a9c2 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:02.638115 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-eb47dcd6-098c-44f1-8fef-e45eb420a9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:02.638700 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ee7dd-4c49-4be1-93bb-a838149ab2b3 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:02.705884 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ee7dd-4c49-4be1-93bb-a838149ab2b3 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:02.710911 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d5a2de-abe5-49f9-a9c1-3c2a83eb17d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:02.711572 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d5a2de-abe5-49f9-a9c1-3c2a83eb17d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:02.711852 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d5a2de-abe5-49f9-a9c1-3c2a83eb17d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:02.712154 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d5a2de-abe5-49f9-a9c1-3c2a83eb17d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:02.712846 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d5a2de-abe5-49f9-a9c1-3c2a83eb17d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:02.714862 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d5a2de-abe5-49f9-a9c1-3c2a83eb17d3 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:02.723659 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb47dcd6-098c-44f1-8fef-e45eb420a9c2 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:02.727646 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:02.750471 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1344/2696] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:46:02 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:46:02.750471 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb930b33-16f7-4738-ba60-31d33966da00 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:02.750471 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb930b33-16f7-4738-ba60-31d33966da00 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:02.750471 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb930b33-16f7-4738-ba60-31d33966da00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:02.781922 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-eb930b33-16f7-4738-ba60-31d33966da00 None None] MFA Rules not processed for user `b65a1f14f9bc46c59214ed240316aa86`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:02.783192 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-eb930b33-16f7-4738-ba60-31d33966da00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:02.813969 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dca9c3-81cf-4b9c-ae38-77f688a1523a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb0ec2ef6c4748e181f1963e7dfe30b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:02.814936 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dca9c3-81cf-4b9c-ae38-77f688a1523a service placement] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:02.815220 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dca9c3-81cf-4b9c-ae38-77f688a1523a service placement] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:02.815492 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dca9c3-81cf-4b9c-ae38-77f688a1523a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:02.817473 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-f0dca9c3-81cf-4b9c-ae38-77f688a1523a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:02.837768 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dca9c3-81cf-4b9c-ae38-77f688a1523a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:02.840112 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dca9c3-81cf-4b9c-ae38-77f688a1523a service placement] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:02.855882 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1353/2697] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:46:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:46:02.857721 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dca9c3-81cf-4b9c-ae38-77f688a1523a service placement] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:02.867187 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1345/2698] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:46:02.878373 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dca9c3-81cf-4b9c-ae38-77f688a1523a service placement] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:02.880628 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb930b33-16f7-4738-ba60-31d33966da00 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:02.884756 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-eb930b33-16f7-4738-ba60-31d33966da00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:02.943480 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:02.968161 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb930b33-16f7-4738-ba60-31d33966da00 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:02.972582 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacba7c9-a846-40e5-a1bb-aaf7b1fad82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:02.973282 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacba7c9-a846-40e5-a1bb-aaf7b1fad82c admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:02.973570 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacba7c9-a846-40e5-a1bb-aaf7b1fad82c admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:02.973852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacba7c9-a846-40e5-a1bb-aaf7b1fad82c admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:02.975881 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8f5012-e9fe-43c2-9d42-ae04927c08e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:02.976960 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacba7c9-a846-40e5-a1bb-aaf7b1fad82c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:02.977297 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f5012-e9fe-43c2-9d42-ae04927c08e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:02.977974 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f5012-e9fe-43c2-9d42-ae04927c08e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:02.978530 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f5012-e9fe-43c2-9d42-ae04927c08e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:02.979106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacba7c9-a846-40e5-a1bb-aaf7b1fad82c admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:02.997180 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8f5012-e9fe-43c2-9d42-ae04927c08e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:03.000199 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8f5012-e9fe-43c2-9d42-ae04927c08e5 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:03.008181 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1354/2699] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:46:02 2022] POST /identity/v3/users => generated 293 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:46:03.013261 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacba7c9-a846-40e5-a1bb-aaf7b1fad82c admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:03.029801 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1346/2700] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:46:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:46:03.031542 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacba7c9-a846-40e5-a1bb-aaf7b1fad82c admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:03.046345 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69cbaf1-4391-436d-8b61-c56776a90ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:03.046345 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69cbaf1-4391-436d-8b61-c56776a90ed7 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:03.046345 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69cbaf1-4391-436d-8b61-c56776a90ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:03.046345 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69cbaf1-4391-436d-8b61-c56776a90ed7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:03.046789 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69cbaf1-4391-436d-8b61-c56776a90ed7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:03.046789 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69cbaf1-4391-436d-8b61-c56776a90ed7 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:03.057970 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1355/2701] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:46:03.062789 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8f5012-e9fe-43c2-9d42-ae04927c08e5 service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:03.073866 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8f5012-e9fe-43c2-9d42-ae04927c08e5 service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:03.078624 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9015faf0-0bca-4c55-a28d-bcbb168d8215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:03.079512 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015faf0-0bca-4c55-a28d-bcbb168d8215 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:03.079920 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015faf0-0bca-4c55-a28d-bcbb168d8215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:03.080208 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015faf0-0bca-4c55-a28d-bcbb168d8215 admin admin] PATH_INFO: `/v3/projects/df421aa84b254cd1924015f0b8f5fc37/users/88a377f15c86466b9500293964b2287a/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:03.096655 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9015faf0-0bca-4c55-a28d-bcbb168d8215 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df421aa84b254cd1924015f0b8f5fc37, user_id=88a377f15c86466b9500293964b2287a, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:03.099382 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9015faf0-0bca-4c55-a28d-bcbb168d8215 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:03.114550 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1347/2702] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:46:03 2022] PUT /identity/v3/projects/df421aa84b254cd1924015f0b8f5fc37/users/88a377f15c86466b9500293964b2287a/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:03.119203 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69cbaf1-4391-436d-8b61-c56776a90ed7 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:03.133008 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69cbaf1-4391-436d-8b61-c56776a90ed7 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:03.204748 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9980b5-aa4d-47f9-bd67-67a15c1919a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:03.205602 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9980b5-aa4d-47f9-bd67-67a15c1919a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:03.205989 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9980b5-aa4d-47f9-bd67-67a15c1919a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:03.206422 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9980b5-aa4d-47f9-bd67-67a15c1919a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:03.207386 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9980b5-aa4d-47f9-bd67-67a15c1919a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:03.209951 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9980b5-aa4d-47f9-bd67-67a15c1919a0 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:03.239099 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1356/2703] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:46:03.239365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9015faf0-0bca-4c55-a28d-bcbb168d8215 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:03.256760 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9015faf0-0bca-4c55-a28d-bcbb168d8215 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:03.267742 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be7e6b9-55e1-4409-b735-c0193dc76804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:03.269278 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7e6b9-55e1-4409-b735-c0193dc76804 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:03.269969 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7e6b9-55e1-4409-b735-c0193dc76804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:03.270564 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7e6b9-55e1-4409-b735-c0193dc76804 admin admin] PATH_INFO: `/v3/projects/df421aa84b254cd1924015f0b8f5fc37/users/88a377f15c86466b9500293964b2287a/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:03.274048 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be7e6b9-55e1-4409-b735-c0193dc76804 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df421aa84b254cd1924015f0b8f5fc37, user_id=88a377f15c86466b9500293964b2287a, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:03.280174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be7e6b9-55e1-4409-b735-c0193dc76804 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:03.301232 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1348/2704] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:46:03 2022] PUT /identity/v3/projects/df421aa84b254cd1924015f0b8f5fc37/users/88a377f15c86466b9500293964b2287a/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:46:03.317535 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596ec22-cee9-452f-ad8d-970f7bc17f0e None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:03.318183 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596ec22-cee9-452f-ad8d-970f7bc17f0e None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:03.318652 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596ec22-cee9-452f-ad8d-970f7bc17f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:03.350278 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:03.369557 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-0596ec22-cee9-452f-ad8d-970f7bc17f0e None None] MFA Rules not processed for user `88a377f15c86466b9500293964b2287a`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:03.371594 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-0596ec22-cee9-452f-ad8d-970f7bc17f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:03.537475 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1357/2705] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:03 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:46:03.553546 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be7e6b9-55e1-4409-b735-c0193dc76804 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:03.626312 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be7e6b9-55e1-4409-b735-c0193dc76804 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:03.630951 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3fa4f-0712-4bbd-830b-6d63642985e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:03.631767 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3fa4f-0712-4bbd-830b-6d63642985e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:03.632241 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3fa4f-0712-4bbd-830b-6d63642985e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:03.632526 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3fa4f-0712-4bbd-830b-6d63642985e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:03.643949 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e3fa4f-0712-4bbd-830b-6d63642985e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:03.645421 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e3fa4f-0712-4bbd-830b-6d63642985e8 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:03.657184 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1349/2706] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:46:03 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:46:04.111814 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0596ec22-cee9-452f-ad8d-970f7bc17f0e None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:04.205764 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0596ec22-cee9-452f-ad8d-970f7bc17f0e None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:04.211741 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e05d18b-9d5e-461c-9fdf-2c026d945bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:04.212933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05d18b-9d5e-461c-9fdf-2c026d945bce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:04.213252 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05d18b-9d5e-461c-9fdf-2c026d945bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:04.213826 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05d18b-9d5e-461c-9fdf-2c026d945bce admin admin] PATH_INFO: `/v3/users/944db5934ce2464992caf38fabd85db6` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:04.215861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e05d18b-9d5e-461c-9fdf-2c026d945bce admin admin] RBAC: Authorizing `identity:delete_user(user_id=944db5934ce2464992caf38fabd85db6)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:04.218076 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e05d18b-9d5e-461c-9fdf-2c026d945bce admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:04.284595 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-7e05d18b-9d5e-461c-9fdf-2c026d945bce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '944db5934ce2464992caf38fabd85db6'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:04.285489 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-7e05d18b-9d5e-461c-9fdf-2c026d945bce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '944db5934ce2464992caf38fabd85db6'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:04.295639 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-7e05d18b-9d5e-461c-9fdf-2c026d945bce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '944db5934ce2464992caf38fabd85db6'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:04.296281 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-7e05d18b-9d5e-461c-9fdf-2c026d945bce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '944db5934ce2464992caf38fabd85db6'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:04.308249 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-7e05d18b-9d5e-461c-9fdf-2c026d945bce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '944db5934ce2464992caf38fabd85db6'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:04.341822 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1358/2707] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:46:04 2022] DELETE /identity/v3/users/944db5934ce2464992caf38fabd85db6 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:04.346303 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a461021-385a-4bb1-afd6-0ac85d5ed3d0 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:04.346886 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a461021-385a-4bb1-afd6-0ac85d5ed3d0 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:04.346993 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a461021-385a-4bb1-afd6-0ac85d5ed3d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:04.370614 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-7a461021-385a-4bb1-afd6-0ac85d5ed3d0 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:04.371853 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7a461021-385a-4bb1-afd6-0ac85d5ed3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:04.449423 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1350/2708] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:46:04.464632 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e05d18b-9d5e-461c-9fdf-2c026d945bce admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:04.466426 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-7e05d18b-9d5e-461c-9fdf-2c026d945bce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:04.507871 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:04.586264 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783dbebb-4be1-4734-ba0e-03a6f6b1e64b None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:04.586561 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783dbebb-4be1-4734-ba0e-03a6f6b1e64b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:04.586850 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783dbebb-4be1-4734-ba0e-03a6f6b1e64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:04.594251 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e05d18b-9d5e-461c-9fdf-2c026d945bce admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:04.601018 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779da3a-d89d-485a-b8b9-56c959044344 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:04.602464 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779da3a-d89d-485a-b8b9-56c959044344 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:04.602648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779da3a-d89d-485a-b8b9-56c959044344 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:04.603035 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779da3a-d89d-485a-b8b9-56c959044344 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:04.613983 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-783dbebb-4be1-4734-ba0e-03a6f6b1e64b None None] MFA Rules not processed for user `b65a1f14f9bc46c59214ed240316aa86`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:04.616398 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-783dbebb-4be1-4734-ba0e-03a6f6b1e64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:04.617023 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e779da3a-d89d-485a-b8b9-56c959044344 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:04.618880 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e779da3a-d89d-485a-b8b9-56c959044344 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:04.640823 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1359/2709] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:46:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:46:04.713910 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:04.735299 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1351/2710] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:46:05.011493 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779da3a-d89d-485a-b8b9-56c959044344 service neutron] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:05.015149 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-e779da3a-d89d-485a-b8b9-56c959044344 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:05.036277 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779da3a-d89d-485a-b8b9-56c959044344 service neutron] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:05.045363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb314672-1bff-4dc9-9917-3ba123ad1e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:05.046924 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb314672-1bff-4dc9-9917-3ba123ad1e75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:05.047388 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb314672-1bff-4dc9-9917-3ba123ad1e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:05.047838 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb314672-1bff-4dc9-9917-3ba123ad1e75 admin admin] PATH_INFO: `/v3/projects/e0b3cf123a284b7ba783dea56e84581e` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:05.051481 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb314672-1bff-4dc9-9917-3ba123ad1e75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0b3cf123a284b7ba783dea56e84581e)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:05.055418 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb314672-1bff-4dc9-9917-3ba123ad1e75 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:05.117381 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-cb314672-1bff-4dc9-9917-3ba123ad1e75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0b3cf123a284b7ba783dea56e84581e', 'request_id': 'req-cb314672-1bff-4dc9-9917-3ba123ad1e75'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:05.128318 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-cb314672-1bff-4dc9-9917-3ba123ad1e75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0b3cf123a284b7ba783dea56e84581e', 'request_id': 'req-cb314672-1bff-4dc9-9917-3ba123ad1e75'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:05.135835 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-cb314672-1bff-4dc9-9917-3ba123ad1e75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0b3cf123a284b7ba783dea56e84581e', 'request_id': 'req-cb314672-1bff-4dc9-9917-3ba123ad1e75'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:05.151910 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-cb314672-1bff-4dc9-9917-3ba123ad1e75 admin admin] The token cache is being invalidate because project e0b3cf123a284b7ba783dea56e84581e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:46:05.152248 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-cb314672-1bff-4dc9-9917-3ba123ad1e75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:05.152876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-cb314672-1bff-4dc9-9917-3ba123ad1e75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:05.154829 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1360/2711] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:46:05 2022] DELETE /identity/v3/projects/e0b3cf123a284b7ba783dea56e84581e => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:05.160691 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783dbebb-4be1-4734-ba0e-03a6f6b1e64b None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:05.174219 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783dbebb-4be1-4734-ba0e-03a6f6b1e64b None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:05.241317 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbcc1d3-3366-43a6-af05-884a1cc0197a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:05.244170 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcc1d3-3366-43a6-af05-884a1cc0197a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:05.244170 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcc1d3-3366-43a6-af05-884a1cc0197a admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:05.244170 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcc1d3-3366-43a6-af05-884a1cc0197a admin admin] PATH_INFO: `/v3/users/88a377f15c86466b9500293964b2287a` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:05.244170 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbcc1d3-3366-43a6-af05-884a1cc0197a admin admin] RBAC: Authorizing `identity:delete_user(user_id=88a377f15c86466b9500293964b2287a)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:05.246694 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbcc1d3-3366-43a6-af05-884a1cc0197a admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:05.313851 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-bfbcc1d3-3366-43a6-af05-884a1cc0197a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88a377f15c86466b9500293964b2287a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:05.314738 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-bfbcc1d3-3366-43a6-af05-884a1cc0197a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88a377f15c86466b9500293964b2287a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:05.315482 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-bfbcc1d3-3366-43a6-af05-884a1cc0197a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88a377f15c86466b9500293964b2287a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:05.341044 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-bfbcc1d3-3366-43a6-af05-884a1cc0197a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88a377f15c86466b9500293964b2287a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:05.350889 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-bfbcc1d3-3366-43a6-af05-884a1cc0197a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88a377f15c86466b9500293964b2287a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:05.372135 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1352/2712] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:46:05 2022] DELETE /identity/v3/users/88a377f15c86466b9500293964b2287a => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:05.726206 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb314672-1bff-4dc9-9917-3ba123ad1e75 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:05.729445 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-cb314672-1bff-4dc9-9917-3ba123ad1e75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:05.747440 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb314672-1bff-4dc9-9917-3ba123ad1e75 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:05.840212 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf4ffb-feb7-4986-994a-2c57dd481533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:05.842270 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf4ffb-feb7-4986-994a-2c57dd481533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:05.842544 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf4ffb-feb7-4986-994a-2c57dd481533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:05.842825 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf4ffb-feb7-4986-994a-2c57dd481533 admin admin] PATH_INFO: `/v3/projects/df421aa84b254cd1924015f0b8f5fc37` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:05.844093 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cf4ffb-feb7-4986-994a-2c57dd481533 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df421aa84b254cd1924015f0b8f5fc37)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:05.846013 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cf4ffb-feb7-4986-994a-2c57dd481533 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:05.896650 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:05.901949 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-78cf4ffb-feb7-4986-994a-2c57dd481533 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df421aa84b254cd1924015f0b8f5fc37', 'request_id': 'req-78cf4ffb-feb7-4986-994a-2c57dd481533'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:05.910388 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-78cf4ffb-feb7-4986-994a-2c57dd481533 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df421aa84b254cd1924015f0b8f5fc37', 'request_id': 'req-78cf4ffb-feb7-4986-994a-2c57dd481533'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:05.921201 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-78cf4ffb-feb7-4986-994a-2c57dd481533 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df421aa84b254cd1924015f0b8f5fc37', 'request_id': 'req-78cf4ffb-feb7-4986-994a-2c57dd481533'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:05.932585 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-78cf4ffb-feb7-4986-994a-2c57dd481533 admin admin] The token cache is being invalidate because project df421aa84b254cd1924015f0b8f5fc37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:46:05.932873 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-78cf4ffb-feb7-4986-994a-2c57dd481533 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:05.933543 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-78cf4ffb-feb7-4986-994a-2c57dd481533 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:05.935198 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1361/2713] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:46:05 2022] DELETE /identity/v3/projects/df421aa84b254cd1924015f0b8f5fc37 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:46:05.940283 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571a9aa-fb2c-4b78-b1ec-c3748ff4cfa0 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:05.940577 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571a9aa-fb2c-4b78-b1ec-c3748ff4cfa0 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:05.940855 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571a9aa-fb2c-4b78-b1ec-c3748ff4cfa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:05.959240 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-d571a9aa-fb2c-4b78-b1ec-c3748ff4cfa0 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:05.960661 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d571a9aa-fb2c-4b78-b1ec-c3748ff4cfa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:06.032386 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1353/2714] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:46:06.043352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a271434-0824-413a-813e-078ff2f5205b None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:06.043813 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a271434-0824-413a-813e-078ff2f5205b None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:06.044267 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a271434-0824-413a-813e-078ff2f5205b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:06.073074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-7a271434-0824-413a-813e-078ff2f5205b None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:06.074340 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-7a271434-0824-413a-813e-078ff2f5205b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:06.082918 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1362/2715] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:46:06.087841 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d571a9aa-fb2c-4b78-b1ec-c3748ff4cfa0 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:06.098234 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d571a9aa-fb2c-4b78-b1ec-c3748ff4cfa0 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:06.102993 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5f65f2-7749-4564-8dda-8716a292d454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:06.103670 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f65f2-7749-4564-8dda-8716a292d454 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:06.104564 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f65f2-7749-4564-8dda-8716a292d454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:06.104840 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f65f2-7749-4564-8dda-8716a292d454 admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:06.105587 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5f65f2-7749-4564-8dda-8716a292d454 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:46:06.105893 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5f65f2-7749-4564-8dda-8716a292d454 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:06.107568 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5f65f2-7749-4564-8dda-8716a292d454 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:06.120307 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1354/2716] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:46:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:46:06.125246 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a271434-0824-413a-813e-078ff2f5205b None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:06.134647 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a271434-0824-413a-813e-078ff2f5205b None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:06.143031 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cca609-46ff-4b48-87f5-ffe02464aff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:06.144751 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cca609-46ff-4b48-87f5-ffe02464aff4 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:06.145963 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cca609-46ff-4b48-87f5-ffe02464aff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:06.146958 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cca609-46ff-4b48-87f5-ffe02464aff4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:06.147684 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cca609-46ff-4b48-87f5-ffe02464aff4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:06.149634 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cca609-46ff-4b48-87f5-ffe02464aff4 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:06.184293 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1363/2717] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:46:06 2022] POST /identity/v3/projects => generated 365 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:46:06.188636 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5f65f2-7749-4564-8dda-8716a292d454 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:06.193926 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:06.199690 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5f65f2-7749-4564-8dda-8716a292d454 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:06.265199 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eb9626-d23c-4d4a-9334-f94ed3d19ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:06.265934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb9626-d23c-4d4a-9334-f94ed3d19ad6 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:06.266286 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb9626-d23c-4d4a-9334-f94ed3d19ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:06.266577 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb9626-d23c-4d4a-9334-f94ed3d19ad6 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:06.267290 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eb9626-d23c-4d4a-9334-f94ed3d19ad6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:06.269254 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eb9626-d23c-4d4a-9334-f94ed3d19ad6 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:06.292587 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1355/2718] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:46:06 2022] POST /identity/v3/users => generated 283 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:46:06.297401 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cca609-46ff-4b48-87f5-ffe02464aff4 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:06.308631 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cca609-46ff-4b48-87f5-ffe02464aff4 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:06.314921 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f39e7-16d3-4747-bcef-f9affdcca1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:06.315684 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f39e7-16d3-4747-bcef-f9affdcca1be admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:06.316004 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f39e7-16d3-4747-bcef-f9affdcca1be admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:06.316289 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f39e7-16d3-4747-bcef-f9affdcca1be admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:06.317084 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f39e7-16d3-4747-bcef-f9affdcca1be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:06.318784 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f39e7-16d3-4747-bcef-f9affdcca1be admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:06.331705 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1364/2719] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:46:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:46:06.336024 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eb9626-d23c-4d4a-9334-f94ed3d19ad6 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:06.347585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eb9626-d23c-4d4a-9334-f94ed3d19ad6 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:06.352671 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e96e47d-2bec-49a2-811e-ee43463011e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:06.353376 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96e47d-2bec-49a2-811e-ee43463011e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:06.353697 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96e47d-2bec-49a2-811e-ee43463011e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:06.353947 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96e47d-2bec-49a2-811e-ee43463011e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:06.354763 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e96e47d-2bec-49a2-811e-ee43463011e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:06.356464 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e96e47d-2bec-49a2-811e-ee43463011e2 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:06.367568 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1356/2720] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:46:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:46:06.371781 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f39e7-16d3-4747-bcef-f9affdcca1be admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:06.383708 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f39e7-16d3-4747-bcef-f9affdcca1be admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:06.389306 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00480c85-70df-4dcc-96c8-88f1cdf847c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:06.390301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00480c85-70df-4dcc-96c8-88f1cdf847c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:06.390549 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00480c85-70df-4dcc-96c8-88f1cdf847c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:06.390833 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00480c85-70df-4dcc-96c8-88f1cdf847c2 admin admin] PATH_INFO: `/v3/projects/6592d3f06bf9497798705efe31ee7bfc/users/485a46c52f2746d1bca8a9b945285d68/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:06.407943 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00480c85-70df-4dcc-96c8-88f1cdf847c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6592d3f06bf9497798705efe31ee7bfc, user_id=485a46c52f2746d1bca8a9b945285d68, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:06.410780 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00480c85-70df-4dcc-96c8-88f1cdf847c2 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:06.425762 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1365/2721] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:46:06 2022] PUT /identity/v3/projects/6592d3f06bf9497798705efe31ee7bfc/users/485a46c52f2746d1bca8a9b945285d68/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:06.436426 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6bd0a0-4ed8-4d90-abc9-d7122d43f011 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:06.436766 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6bd0a0-4ed8-4d90-abc9-d7122d43f011 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:06.437073 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6bd0a0-4ed8-4d90-abc9-d7122d43f011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:06.467240 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-ce6bd0a0-4ed8-4d90-abc9-d7122d43f011 None None] MFA Rules not processed for user `485a46c52f2746d1bca8a9b945285d68`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:06.468513 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ce6bd0a0-4ed8-4d90-abc9-d7122d43f011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:06.554507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1357/2722] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:06 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:46:06.559302 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00480c85-70df-4dcc-96c8-88f1cdf847c2 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:06.632217 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00480c85-70df-4dcc-96c8-88f1cdf847c2 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:06.638554 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f628d-2aa7-46c5-85b0-4cb31688ca97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:06.639296 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f628d-2aa7-46c5-85b0-4cb31688ca97 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:06.639617 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f628d-2aa7-46c5-85b0-4cb31688ca97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:06.639933 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f628d-2aa7-46c5-85b0-4cb31688ca97 admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:06.640696 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000f628d-2aa7-46c5-85b0-4cb31688ca97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:06.642859 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000f628d-2aa7-46c5-85b0-4cb31688ca97 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:06.676135 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1366/2723] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:46:06 2022] POST /identity/v3/projects => generated 365 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:46:06.679821 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6bd0a0-4ed8-4d90-abc9-d7122d43f011 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:06.692469 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6bd0a0-4ed8-4d90-abc9-d7122d43f011 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:06.766708 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd311c0-44d0-4241-8939-3ed99d6b2843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:06.768064 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd311c0-44d0-4241-8939-3ed99d6b2843 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:06.768253 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd311c0-44d0-4241-8939-3ed99d6b2843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:06.768566 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd311c0-44d0-4241-8939-3ed99d6b2843 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:06.769646 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd311c0-44d0-4241-8939-3ed99d6b2843 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:06.772823 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd311c0-44d0-4241-8939-3ed99d6b2843 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:06.793680 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:06.802065 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1358/2724] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:46:06 2022] POST /identity/v3/users => generated 289 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:46:06.806471 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f628d-2aa7-46c5-85b0-4cb31688ca97 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:06.820667 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f628d-2aa7-46c5-85b0-4cb31688ca97 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:06.829915 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513a8b6-5804-4624-8726-4db467a2c385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:06.830211 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513a8b6-5804-4624-8726-4db467a2c385 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:06.830664 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513a8b6-5804-4624-8726-4db467a2c385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:06.831107 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513a8b6-5804-4624-8726-4db467a2c385 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:06.832339 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a513a8b6-5804-4624-8726-4db467a2c385 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:06.834661 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a513a8b6-5804-4624-8726-4db467a2c385 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:06.851422 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1367/2725] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:46:06 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:46:06.856746 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd311c0-44d0-4241-8939-3ed99d6b2843 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:06.871139 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd311c0-44d0-4241-8939-3ed99d6b2843 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:06.888176 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b61c0-c1f9-46b4-acc4-c680fab41a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:06.889776 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b61c0-c1f9-46b4-acc4-c680fab41a9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:06.890369 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b61c0-c1f9-46b4-acc4-c680fab41a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:06.890871 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b61c0-c1f9-46b4-acc4-c680fab41a9f admin admin] PATH_INFO: `/v3/projects/e2ffc1aa52744f32be65865c165f5ca5/users/7df48f4accc74a67b41ea74b1956d413/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:06.913152 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b61c0-c1f9-46b4-acc4-c680fab41a9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2ffc1aa52744f32be65865c165f5ca5, user_id=7df48f4accc74a67b41ea74b1956d413, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:06.916015 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b61c0-c1f9-46b4-acc4-c680fab41a9f admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:06.932677 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1359/2726] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:46:06 2022] PUT /identity/v3/projects/e2ffc1aa52744f32be65865c165f5ca5/users/7df48f4accc74a67b41ea74b1956d413/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:06.937068 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513a8b6-5804-4624-8726-4db467a2c385 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:06.951916 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513a8b6-5804-4624-8726-4db467a2c385 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:07.007837 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:07.034099 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c92789-c5d1-4726-99ff-89174a7dea64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:07.035701 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c92789-c5d1-4726-99ff-89174a7dea64 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:07.036745 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c92789-c5d1-4726-99ff-89174a7dea64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:07.037103 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c92789-c5d1-4726-99ff-89174a7dea64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:07.038984 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c92789-c5d1-4726-99ff-89174a7dea64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:07.041623 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c92789-c5d1-4726-99ff-89174a7dea64 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:07.053601 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1368/2727] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:46:06 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:46:07.058939 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b61c0-c1f9-46b4-acc4-c680fab41a9f admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:07.069600 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b61c0-c1f9-46b4-acc4-c680fab41a9f admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:07.074468 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58191d01-63cc-4094-9b12-5322f6d86cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:07.075376 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58191d01-63cc-4094-9b12-5322f6d86cc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:07.075659 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58191d01-63cc-4094-9b12-5322f6d86cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:07.076492 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58191d01-63cc-4094-9b12-5322f6d86cc0 admin admin] PATH_INFO: `/v3/projects/e2ffc1aa52744f32be65865c165f5ca5/users/7df48f4accc74a67b41ea74b1956d413/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:07.078126 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58191d01-63cc-4094-9b12-5322f6d86cc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2ffc1aa52744f32be65865c165f5ca5, user_id=7df48f4accc74a67b41ea74b1956d413, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:07.080841 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58191d01-63cc-4094-9b12-5322f6d86cc0 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:07.092835 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1360/2728] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:46:07 2022] PUT /identity/v3/projects/e2ffc1aa52744f32be65865c165f5ca5/users/7df48f4accc74a67b41ea74b1956d413/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:07.106590 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4a6d1-efa4-4def-bef4-a0800d841ba1 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:07.106884 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4a6d1-efa4-4def-bef4-a0800d841ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:07.107171 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4a6d1-efa4-4def-bef4-a0800d841ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:07.137469 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-05d4a6d1-efa4-4def-bef4-a0800d841ba1 None None] MFA Rules not processed for user `7df48f4accc74a67b41ea74b1956d413`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:07.138729 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-05d4a6d1-efa4-4def-bef4-a0800d841ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:07.219324 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1369/2729] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:07 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:46:07.233129 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58191d01-63cc-4094-9b12-5322f6d86cc0 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:07.234966 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-58191d01-63cc-4094-9b12-5322f6d86cc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:07.301127 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58191d01-63cc-4094-9b12-5322f6d86cc0 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:07.305129 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce3aed2-87b3-4297-8170-1a0dd1f2a77d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:07.305801 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3aed2-87b3-4297-8170-1a0dd1f2a77d service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:07.306080 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3aed2-87b3-4297-8170-1a0dd1f2a77d service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:07.306393 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3aed2-87b3-4297-8170-1a0dd1f2a77d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:07.317416 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce3aed2-87b3-4297-8170-1a0dd1f2a77d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:07.319048 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce3aed2-87b3-4297-8170-1a0dd1f2a77d service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:07.330764 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1361/2730] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:46:07 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:46:07.830991 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d4a6d1-efa4-4def-bef4-a0800d841ba1 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:07.911706 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d4a6d1-efa4-4def-bef4-a0800d841ba1 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:07.918836 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c8474-f86b-44cb-98b6-446715c7c6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:07.920292 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c8474-f86b-44cb-98b6-446715c7c6fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:07.920773 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c8474-f86b-44cb-98b6-446715c7c6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:07.921220 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c8474-f86b-44cb-98b6-446715c7c6fa admin admin] PATH_INFO: `/v3/users/485a46c52f2746d1bca8a9b945285d68` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:07.923074 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01c8474-f86b-44cb-98b6-446715c7c6fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=485a46c52f2746d1bca8a9b945285d68)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:07.926536 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01c8474-f86b-44cb-98b6-446715c7c6fa admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:07.995394 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-b01c8474-f86b-44cb-98b6-446715c7c6fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '485a46c52f2746d1bca8a9b945285d68'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:07.996085 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-b01c8474-f86b-44cb-98b6-446715c7c6fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '485a46c52f2746d1bca8a9b945285d68'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:08.004713 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-b01c8474-f86b-44cb-98b6-446715c7c6fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '485a46c52f2746d1bca8a9b945285d68'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:08.005659 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-b01c8474-f86b-44cb-98b6-446715c7c6fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '485a46c52f2746d1bca8a9b945285d68'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:08.013806 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-b01c8474-f86b-44cb-98b6-446715c7c6fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '485a46c52f2746d1bca8a9b945285d68'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:08.016475 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:08.047967 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1370/2731] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:46:07 2022] DELETE /identity/v3/users/485a46c52f2746d1bca8a9b945285d68 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:08.055201 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae23d52-5182-4e13-aef2-a943e24170fc None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:08.055801 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae23d52-5182-4e13-aef2-a943e24170fc None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:08.055801 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae23d52-5182-4e13-aef2-a943e24170fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:08.080102 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-eae23d52-5182-4e13-aef2-a943e24170fc None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:08.081329 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-eae23d52-5182-4e13-aef2-a943e24170fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:08.152595 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1362/2732] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:46:08.165082 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c8474-f86b-44cb-98b6-446715c7c6fa admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:08.167331 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-b01c8474-f86b-44cb-98b6-446715c7c6fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:08.260058 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c8474-f86b-44cb-98b6-446715c7c6fa admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:08.267097 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae3e2fc-ad6a-436c-8644-b4dedfc80789 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:08.267855 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3e2fc-ad6a-436c-8644-b4dedfc80789 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:08.268136 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3e2fc-ad6a-436c-8644-b4dedfc80789 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:08.268582 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3e2fc-ad6a-436c-8644-b4dedfc80789 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:08.285516 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae3e2fc-ad6a-436c-8644-b4dedfc80789 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:08.288000 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae3e2fc-ad6a-436c-8644-b4dedfc80789 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:08.304099 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1371/2733] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:46:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:46:08.640925 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf25d9-a6fd-4855-852d-a649dd48f70e None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:08.641217 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf25d9-a6fd-4855-852d-a649dd48f70e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:08.641500 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf25d9-a6fd-4855-852d-a649dd48f70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:08.662520 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-92cf25d9-a6fd-4855-852d-a649dd48f70e None None] MFA Rules not processed for user `67aa048d409c46e9a729aec9d1b2286e`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:08.663942 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-92cf25d9-a6fd-4855-852d-a649dd48f70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:08.752329 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1363/2734] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Jun 15 19:46:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:46:08.846927 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae3e2fc-ad6a-436c-8644-b4dedfc80789 service neutron] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:08.848409 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-6ae3e2fc-ad6a-436c-8644-b4dedfc80789 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:08.949653 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae3e2fc-ad6a-436c-8644-b4dedfc80789 service neutron] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:08.955264 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e1622-4c3e-43e1-9c50-651ebd748628 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176d5775a70f4524a9b7fe1ce9d426ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:08.955938 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e1622-4c3e-43e1-9c50-651ebd748628 service swift] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:08.957892 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e1622-4c3e-43e1-9c50-651ebd748628 service swift] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:08.957892 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e1622-4c3e-43e1-9c50-651ebd748628 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:08.968676 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878e1622-4c3e-43e1-9c50-651ebd748628 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:08.970598 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878e1622-4c3e-43e1-9c50-651ebd748628 service swift] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:08.980756 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1372/2735] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Jun 15 19:46:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 15 19:46:09.296654 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf25d9-a6fd-4855-852d-a649dd48f70e None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:09.300507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-92cf25d9-a6fd-4855-852d-a649dd48f70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:09.362814 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf25d9-a6fd-4855-852d-a649dd48f70e None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:09.386356 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b7110d-f0b1-481d-87e9-7f0e126e3ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:09.387971 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7110d-f0b1-481d-87e9-7f0e126e3ae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:09.392023 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7110d-f0b1-481d-87e9-7f0e126e3ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:09.392372 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7110d-f0b1-481d-87e9-7f0e126e3ae1 admin admin] PATH_INFO: `/v3/projects/6592d3f06bf9497798705efe31ee7bfc` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:09.393858 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b7110d-f0b1-481d-87e9-7f0e126e3ae1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6592d3f06bf9497798705efe31ee7bfc)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:09.396730 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b7110d-f0b1-481d-87e9-7f0e126e3ae1 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:09.412733 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:09.470950 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e0b7110d-f0b1-481d-87e9-7f0e126e3ae1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6592d3f06bf9497798705efe31ee7bfc', 'request_id': 'req-e0b7110d-f0b1-481d-87e9-7f0e126e3ae1'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:09.483963 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e0b7110d-f0b1-481d-87e9-7f0e126e3ae1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6592d3f06bf9497798705efe31ee7bfc', 'request_id': 'req-e0b7110d-f0b1-481d-87e9-7f0e126e3ae1'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:09.496536 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e0b7110d-f0b1-481d-87e9-7f0e126e3ae1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6592d3f06bf9497798705efe31ee7bfc', 'request_id': 'req-e0b7110d-f0b1-481d-87e9-7f0e126e3ae1'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:09.520900 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e0b7110d-f0b1-481d-87e9-7f0e126e3ae1 admin admin] The token cache is being invalidate because project 6592d3f06bf9497798705efe31ee7bfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:46:09.535833 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e0b7110d-f0b1-481d-87e9-7f0e126e3ae1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:09.535833 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e0b7110d-f0b1-481d-87e9-7f0e126e3ae1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:09.535833 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1364/2736] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:46:09 2022] DELETE /identity/v3/projects/6592d3f06bf9497798705efe31ee7bfc => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:09.536342 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e1622-4c3e-43e1-9c50-651ebd748628 service swift] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:09.544393 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e1622-4c3e-43e1-9c50-651ebd748628 service swift] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:09.632255 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009bbab-7ebf-435b-8738-73e052600df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:09.633336 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009bbab-7ebf-435b-8738-73e052600df7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:09.634002 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009bbab-7ebf-435b-8738-73e052600df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:09.634374 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009bbab-7ebf-435b-8738-73e052600df7 admin admin] PATH_INFO: `/v3/users/7df48f4accc74a67b41ea74b1956d413` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:09.636361 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d009bbab-7ebf-435b-8738-73e052600df7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7df48f4accc74a67b41ea74b1956d413)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:09.638532 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d009bbab-7ebf-435b-8738-73e052600df7 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:09.669418 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:09.747726 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-d009bbab-7ebf-435b-8738-73e052600df7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7df48f4accc74a67b41ea74b1956d413'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:09.750216 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-d009bbab-7ebf-435b-8738-73e052600df7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7df48f4accc74a67b41ea74b1956d413'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:09.762905 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-d009bbab-7ebf-435b-8738-73e052600df7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7df48f4accc74a67b41ea74b1956d413'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:09.763756 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-d009bbab-7ebf-435b-8738-73e052600df7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7df48f4accc74a67b41ea74b1956d413'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:09.776157 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-d009bbab-7ebf-435b-8738-73e052600df7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7df48f4accc74a67b41ea74b1956d413'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:09.807812 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1373/2737] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:46:09 2022] DELETE /identity/v3/users/7df48f4accc74a67b41ea74b1956d413 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:10.471045 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b7110d-f0b1-481d-87e9-7f0e126e3ae1 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:10.473399 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e0b7110d-f0b1-481d-87e9-7f0e126e3ae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:10.496925 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b7110d-f0b1-481d-87e9-7f0e126e3ae1 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:10.638155 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22457cac-47bd-43f8-8cb9-7f39ef6f2d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:10.639061 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22457cac-47bd-43f8-8cb9-7f39ef6f2d1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:10.640081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22457cac-47bd-43f8-8cb9-7f39ef6f2d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:10.640370 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22457cac-47bd-43f8-8cb9-7f39ef6f2d1e admin admin] PATH_INFO: `/v3/projects/e2ffc1aa52744f32be65865c165f5ca5` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:10.641865 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22457cac-47bd-43f8-8cb9-7f39ef6f2d1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2ffc1aa52744f32be65865c165f5ca5)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:10.646030 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22457cac-47bd-43f8-8cb9-7f39ef6f2d1e admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:10.714478 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-22457cac-47bd-43f8-8cb9-7f39ef6f2d1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2ffc1aa52744f32be65865c165f5ca5', 'request_id': 'req-22457cac-47bd-43f8-8cb9-7f39ef6f2d1e'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:10.743990 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-22457cac-47bd-43f8-8cb9-7f39ef6f2d1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2ffc1aa52744f32be65865c165f5ca5', 'request_id': 'req-22457cac-47bd-43f8-8cb9-7f39ef6f2d1e'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:10.761732 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-22457cac-47bd-43f8-8cb9-7f39ef6f2d1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2ffc1aa52744f32be65865c165f5ca5', 'request_id': 'req-22457cac-47bd-43f8-8cb9-7f39ef6f2d1e'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:10.782032 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-22457cac-47bd-43f8-8cb9-7f39ef6f2d1e admin admin] The token cache is being invalidate because project e2ffc1aa52744f32be65865c165f5ca5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:46:10.782360 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-22457cac-47bd-43f8-8cb9-7f39ef6f2d1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:10.784385 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-22457cac-47bd-43f8-8cb9-7f39ef6f2d1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:10.786533 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1365/2738] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:46:10 2022] DELETE /identity/v3/projects/e2ffc1aa52744f32be65865c165f5ca5 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:10.792521 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b1092-8e1d-4ce5-bd2e-3ccf6d5d8081 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:10.792970 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b1092-8e1d-4ce5-bd2e-3ccf6d5d8081 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:10.793080 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b1092-8e1d-4ce5-bd2e-3ccf6d5d8081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:10.819733 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-dd1b1092-8e1d-4ce5-bd2e-3ccf6d5d8081 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:10.821105 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-dd1b1092-8e1d-4ce5-bd2e-3ccf6d5d8081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:10.921487 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1374/2739] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:46:10.923539 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7b08e-9772-4b79-b4ee-e8fee4c835ec None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:10.923866 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7b08e-9772-4b79-b4ee-e8fee4c835ec None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:10.924143 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7b08e-9772-4b79-b4ee-e8fee4c835ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:10.977259 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-87d7b08e-9772-4b79-b4ee-e8fee4c835ec None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:10.987955 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-87d7b08e-9772-4b79-b4ee-e8fee4c835ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:11.020549 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1366/2740] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:46:11.027817 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b1092-8e1d-4ce5-bd2e-3ccf6d5d8081 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:11.053275 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b1092-8e1d-4ce5-bd2e-3ccf6d5d8081 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:11.062490 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1760f522-9fdf-4906-9473-279ec100a41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:11.063586 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760f522-9fdf-4906-9473-279ec100a41d admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:11.064077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760f522-9fdf-4906-9473-279ec100a41d admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:11.064568 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760f522-9fdf-4906-9473-279ec100a41d admin admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:11.065939 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1760f522-9fdf-4906-9473-279ec100a41d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:46:11.066637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1760f522-9fdf-4906-9473-279ec100a41d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:11.075358 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1760f522-9fdf-4906-9473-279ec100a41d admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:11.108517 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1375/2741] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:46:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:46:11.111624 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d7b08e-9772-4b79-b4ee-e8fee4c835ec None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:11.158935 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d7b08e-9772-4b79-b4ee-e8fee4c835ec None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:11.173581 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec4253a-6a93-4487-aaf9-b02561986ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:11.188786 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec4253a-6a93-4487-aaf9-b02561986ccd admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:11.191243 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec4253a-6a93-4487-aaf9-b02561986ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:11.193641 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec4253a-6a93-4487-aaf9-b02561986ccd admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:11.197030 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec4253a-6a93-4487-aaf9-b02561986ccd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:11.209706 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec4253a-6a93-4487-aaf9-b02561986ccd admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:11.272556 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:11.315945 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1367/2742] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:46:11 2022] POST /identity/v3/projects => generated 361 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:46:11.323124 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1760f522-9fdf-4906-9473-279ec100a41d admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:11.369944 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1760f522-9fdf-4906-9473-279ec100a41d admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:11.511648 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008af084-12b4-401f-8332-9841a49e0b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:11.512772 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008af084-12b4-401f-8332-9841a49e0b3c admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:11.513547 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008af084-12b4-401f-8332-9841a49e0b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:11.514036 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008af084-12b4-401f-8332-9841a49e0b3c admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:11.515656 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008af084-12b4-401f-8332-9841a49e0b3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:11.521209 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008af084-12b4-401f-8332-9841a49e0b3c admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:11.616183 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1376/2743] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:46:11 2022] POST /identity/v3/users => generated 281 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:46:11.618063 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec4253a-6a93-4487-aaf9-b02561986ccd admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:11.644005 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec4253a-6a93-4487-aaf9-b02561986ccd admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:11.660613 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf049479-3ab8-4b96-9833-5b551caf16ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:11.661731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf049479-3ab8-4b96-9833-5b551caf16ed admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:11.662219 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf049479-3ab8-4b96-9833-5b551caf16ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:11.662656 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf049479-3ab8-4b96-9833-5b551caf16ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:11.664220 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf049479-3ab8-4b96-9833-5b551caf16ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:11.667890 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf049479-3ab8-4b96-9833-5b551caf16ed admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:11.695474 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1368/2744] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:46:11.698972 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008af084-12b4-401f-8332-9841a49e0b3c admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:11.724272 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008af084-12b4-401f-8332-9841a49e0b3c admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:11.757038 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f8aa3-1147-4522-b955-a79059436dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:11.758169 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f8aa3-1147-4522-b955-a79059436dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:11.768536 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f8aa3-1147-4522-b955-a79059436dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:11.769243 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f8aa3-1147-4522-b955-a79059436dd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:11.770662 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1f8aa3-1147-4522-b955-a79059436dd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:11.773598 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1f8aa3-1147-4522-b955-a79059436dd4 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:11.801447 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:11.821992 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1377/2745] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:46:11.823906 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf049479-3ab8-4b96-9833-5b551caf16ed admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:11.848214 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf049479-3ab8-4b96-9833-5b551caf16ed admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:11.861766 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d07ffd-e1b7-47b3-ae64-00035e0e4a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:11.863435 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d07ffd-e1b7-47b3-ae64-00035e0e4a9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:11.863893 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d07ffd-e1b7-47b3-ae64-00035e0e4a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:11.864387 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d07ffd-e1b7-47b3-ae64-00035e0e4a9b admin admin] PATH_INFO: `/v3/projects/af1f59e9e1dd42058270ed42daf9db73/users/ca2d60ab317a446ebd9868cd3cb8426c/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:11.891178 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d07ffd-e1b7-47b3-ae64-00035e0e4a9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=af1f59e9e1dd42058270ed42daf9db73, user_id=ca2d60ab317a446ebd9868cd3cb8426c, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:11.896845 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d07ffd-e1b7-47b3-ae64-00035e0e4a9b admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:11.929855 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1369/2746] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:46:11 2022] PUT /identity/v3/projects/af1f59e9e1dd42058270ed42daf9db73/users/ca2d60ab317a446ebd9868cd3cb8426c/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:11.944069 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621970e5-02c6-4d2c-ae59-e0fcc08e2fdc None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:11.944561 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621970e5-02c6-4d2c-ae59-e0fcc08e2fdc None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:11.944852 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621970e5-02c6-4d2c-ae59-e0fcc08e2fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:12.020733 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-621970e5-02c6-4d2c-ae59-e0fcc08e2fdc None None] MFA Rules not processed for user `ca2d60ab317a446ebd9868cd3cb8426c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:12.022254 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-621970e5-02c6-4d2c-ae59-e0fcc08e2fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:12.151204 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1378/2747] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:11 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:46:12.160441 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d07ffd-e1b7-47b3-ae64-00035e0e4a9b admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:12.270377 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d07ffd-e1b7-47b3-ae64-00035e0e4a9b admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:12.276379 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6777c38-4446-4ac4-9a43-c01a4ca4fd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:12.277413 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6777c38-4446-4ac4-9a43-c01a4ca4fd9c admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:12.278020 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6777c38-4446-4ac4-9a43-c01a4ca4fd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:12.278533 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6777c38-4446-4ac4-9a43-c01a4ca4fd9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:12.279609 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6777c38-4446-4ac4-9a43-c01a4ca4fd9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:12.281983 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6777c38-4446-4ac4-9a43-c01a4ca4fd9c admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:12.327766 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1370/2748] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:46:12 2022] POST /identity/v3/projects => generated 361 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:46:12.331741 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621970e5-02c6-4d2c-ae59-e0fcc08e2fdc None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:12.354748 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621970e5-02c6-4d2c-ae59-e0fcc08e2fdc None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:12.433255 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99daba88-4631-451c-b94c-525bc0446da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:12.434234 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99daba88-4631-451c-b94c-525bc0446da8 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:12.434645 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99daba88-4631-451c-b94c-525bc0446da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:12.435098 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99daba88-4631-451c-b94c-525bc0446da8 admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:12.436382 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99daba88-4631-451c-b94c-525bc0446da8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:12.440230 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99daba88-4631-451c-b94c-525bc0446da8 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:12.483509 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1379/2749] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:46:12 2022] POST /identity/v3/users => generated 287 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:46:12.487949 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6777c38-4446-4ac4-9a43-c01a4ca4fd9c admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:12.500649 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6777c38-4446-4ac4-9a43-c01a4ca4fd9c admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:12.511237 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3128abd-4705-4271-9b66-15ac028a38dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:12.512325 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3128abd-4705-4271-9b66-15ac028a38dc admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:12.512888 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3128abd-4705-4271-9b66-15ac028a38dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:12.513329 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3128abd-4705-4271-9b66-15ac028a38dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:12.514487 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3128abd-4705-4271-9b66-15ac028a38dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:12.516765 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3128abd-4705-4271-9b66-15ac028a38dc admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:12.534166 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1371/2750] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:46:12.537072 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99daba88-4631-451c-b94c-525bc0446da8 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:12.553067 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99daba88-4631-451c-b94c-525bc0446da8 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:12.562067 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51790d-cedc-4f73-9793-c2dbe0c0e85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:12.563023 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51790d-cedc-4f73-9793-c2dbe0c0e85f admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:12.563308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51790d-cedc-4f73-9793-c2dbe0c0e85f admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:12.563586 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51790d-cedc-4f73-9793-c2dbe0c0e85f admin admin] PATH_INFO: `/v3/projects/5bf2ef9f649542578bb77ac5dd4bd980/users/c151490d652140c198a85ac2c43f9506/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:12.581776 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d51790d-cedc-4f73-9793-c2dbe0c0e85f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bf2ef9f649542578bb77ac5dd4bd980, user_id=c151490d652140c198a85ac2c43f9506, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:12.584662 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d51790d-cedc-4f73-9793-c2dbe0c0e85f admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:12.600917 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1380/2751] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:46:12 2022] PUT /identity/v3/projects/5bf2ef9f649542578bb77ac5dd4bd980/users/c151490d652140c198a85ac2c43f9506/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:46:12.608218 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3128abd-4705-4271-9b66-15ac028a38dc admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:12.621343 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3128abd-4705-4271-9b66-15ac028a38dc admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:12.688894 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28aced1-5c90-4c2c-86ff-edf652cc441d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:12.689486 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28aced1-5c90-4c2c-86ff-edf652cc441d admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:12.689791 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28aced1-5c90-4c2c-86ff-edf652cc441d admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:12.690053 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28aced1-5c90-4c2c-86ff-edf652cc441d admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:12.690817 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28aced1-5c90-4c2c-86ff-edf652cc441d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:12.692447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28aced1-5c90-4c2c-86ff-edf652cc441d admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:12.707390 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1372/2752] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:46:12.710572 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51790d-cedc-4f73-9793-c2dbe0c0e85f admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:12.721865 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51790d-cedc-4f73-9793-c2dbe0c0e85f admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:12.732791 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a1ddd-1aef-4b4f-96c3-ef16fb75b721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:12.732791 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a1ddd-1aef-4b4f-96c3-ef16fb75b721 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:12.732791 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a1ddd-1aef-4b4f-96c3-ef16fb75b721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:12.732791 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a1ddd-1aef-4b4f-96c3-ef16fb75b721 admin admin] PATH_INFO: `/v3/projects/5bf2ef9f649542578bb77ac5dd4bd980/users/c151490d652140c198a85ac2c43f9506/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:12.734057 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030a1ddd-1aef-4b4f-96c3-ef16fb75b721 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bf2ef9f649542578bb77ac5dd4bd980, user_id=c151490d652140c198a85ac2c43f9506, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:12.736908 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030a1ddd-1aef-4b4f-96c3-ef16fb75b721 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:12.748763 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1381/2753] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:46:12 2022] PUT /identity/v3/projects/5bf2ef9f649542578bb77ac5dd4bd980/users/c151490d652140c198a85ac2c43f9506/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:12.759724 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614dbe20-91b1-46e3-aacc-4bc50316f67d None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:12.760024 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614dbe20-91b1-46e3-aacc-4bc50316f67d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:12.760355 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614dbe20-91b1-46e3-aacc-4bc50316f67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:12.791659 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-614dbe20-91b1-46e3-aacc-4bc50316f67d None None] MFA Rules not processed for user `c151490d652140c198a85ac2c43f9506`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:12.801749 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-614dbe20-91b1-46e3-aacc-4bc50316f67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:12.824476 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:12.912048 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1373/2754] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:12 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:46:12.926537 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a1ddd-1aef-4b4f-96c3-ef16fb75b721 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:12.929329 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-030a1ddd-1aef-4b4f-96c3-ef16fb75b721 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:13.054599 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:13.078460 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a1ddd-1aef-4b4f-96c3-ef16fb75b721 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:13.099300 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7668fc0e-dd74-40f5-9eba-ae9581934d49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:13.100407 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668fc0e-dd74-40f5-9eba-ae9581934d49 service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:13.100818 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668fc0e-dd74-40f5-9eba-ae9581934d49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:13.101139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668fc0e-dd74-40f5-9eba-ae9581934d49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:13.122393 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7668fc0e-dd74-40f5-9eba-ae9581934d49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:13.123858 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7668fc0e-dd74-40f5-9eba-ae9581934d49 service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:13.138419 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1382/2755] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:46:12 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:46:13.256974 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614dbe20-91b1-46e3-aacc-4bc50316f67d None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:13.273460 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614dbe20-91b1-46e3-aacc-4bc50316f67d None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:13.284034 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb386d9-2b8c-4cea-a90c-9d33d3112822 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:13.285096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb386d9-2b8c-4cea-a90c-9d33d3112822 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:13.289432 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb386d9-2b8c-4cea-a90c-9d33d3112822 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:13.289942 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb386d9-2b8c-4cea-a90c-9d33d3112822 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:13.375110 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb386d9-2b8c-4cea-a90c-9d33d3112822 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:13.376579 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb386d9-2b8c-4cea-a90c-9d33d3112822 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:13.389061 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1374/2756] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:46:13 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:46:16.570637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7668fc0e-dd74-40f5-9eba-ae9581934d49 service cinder] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:16.682568 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7668fc0e-dd74-40f5-9eba-ae9581934d49 service cinder] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:16.688044 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78076a08-1710-47c1-b2e6-aa273197bd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:16.689214 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78076a08-1710-47c1-b2e6-aa273197bd16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:16.689657 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78076a08-1710-47c1-b2e6-aa273197bd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:16.690125 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78076a08-1710-47c1-b2e6-aa273197bd16 admin admin] PATH_INFO: `/v3/users/ca2d60ab317a446ebd9868cd3cb8426c` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:16.691679 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78076a08-1710-47c1-b2e6-aa273197bd16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca2d60ab317a446ebd9868cd3cb8426c)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:16.694035 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78076a08-1710-47c1-b2e6-aa273197bd16 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:16.775114 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-78076a08-1710-47c1-b2e6-aa273197bd16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca2d60ab317a446ebd9868cd3cb8426c'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:16.776012 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-78076a08-1710-47c1-b2e6-aa273197bd16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca2d60ab317a446ebd9868cd3cb8426c'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:16.788595 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-78076a08-1710-47c1-b2e6-aa273197bd16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca2d60ab317a446ebd9868cd3cb8426c'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:16.789444 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-78076a08-1710-47c1-b2e6-aa273197bd16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca2d60ab317a446ebd9868cd3cb8426c'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:16.800534 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-78076a08-1710-47c1-b2e6-aa273197bd16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca2d60ab317a446ebd9868cd3cb8426c'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:16.825059 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1383/2757] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:46:16 2022] DELETE /identity/v3/users/ca2d60ab317a446ebd9868cd3cb8426c => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:16.829721 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bec72c-c865-43cc-8148-56fadf36370a None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:16.830317 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bec72c-c865-43cc-8148-56fadf36370a None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:16.830979 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bec72c-c865-43cc-8148-56fadf36370a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:16.851742 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-31bec72c-c865-43cc-8148-56fadf36370a None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:16.853915 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-31bec72c-c865-43cc-8148-56fadf36370a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:16.938640 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1375/2758] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:46:16.946235 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78076a08-1710-47c1-b2e6-aa273197bd16 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:16.948423 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-78076a08-1710-47c1-b2e6-aa273197bd16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:17.029971 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78076a08-1710-47c1-b2e6-aa273197bd16 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:17.034440 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b2ac4-3385-4c50-b495-ad96f14ebb21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:17.035200 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b2ac4-3385-4c50-b495-ad96f14ebb21 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:17.035564 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b2ac4-3385-4c50-b495-ad96f14ebb21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:17.035916 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b2ac4-3385-4c50-b495-ad96f14ebb21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:17.048162 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b2ac4-3385-4c50-b495-ad96f14ebb21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:17.049782 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b2ac4-3385-4c50-b495-ad96f14ebb21 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:17.066994 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1384/2759] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:46:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:46:17.442558 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bec72c-c865-43cc-8148-56fadf36370a None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:17.444378 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-31bec72c-c865-43cc-8148-56fadf36370a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:17.479360 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bec72c-c865-43cc-8148-56fadf36370a None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:17.491792 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626a8035-2187-45f0-9349-ca5bd5cf1eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:17.493127 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a8035-2187-45f0-9349-ca5bd5cf1eb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:17.494195 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a8035-2187-45f0-9349-ca5bd5cf1eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:17.494725 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a8035-2187-45f0-9349-ca5bd5cf1eb3 admin admin] PATH_INFO: `/v3/projects/af1f59e9e1dd42058270ed42daf9db73` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:17.497325 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626a8035-2187-45f0-9349-ca5bd5cf1eb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af1f59e9e1dd42058270ed42daf9db73)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:17.500801 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626a8035-2187-45f0-9349-ca5bd5cf1eb3 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:17.621204 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-626a8035-2187-45f0-9349-ca5bd5cf1eb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af1f59e9e1dd42058270ed42daf9db73', 'request_id': 'req-626a8035-2187-45f0-9349-ca5bd5cf1eb3'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:17.623983 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:17.633943 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-626a8035-2187-45f0-9349-ca5bd5cf1eb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af1f59e9e1dd42058270ed42daf9db73', 'request_id': 'req-626a8035-2187-45f0-9349-ca5bd5cf1eb3'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:17.645952 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-626a8035-2187-45f0-9349-ca5bd5cf1eb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af1f59e9e1dd42058270ed42daf9db73', 'request_id': 'req-626a8035-2187-45f0-9349-ca5bd5cf1eb3'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:17.662313 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-626a8035-2187-45f0-9349-ca5bd5cf1eb3 admin admin] The token cache is being invalidate because project af1f59e9e1dd42058270ed42daf9db73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:46:17.662751 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-626a8035-2187-45f0-9349-ca5bd5cf1eb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:17.663779 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-626a8035-2187-45f0-9349-ca5bd5cf1eb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:17.665054 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1376/2760] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:46:17 2022] DELETE /identity/v3/projects/af1f59e9e1dd42058270ed42daf9db73 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:17.669730 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b2ac4-3385-4c50-b495-ad96f14ebb21 service neutron] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:17.681876 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b2ac4-3385-4c50-b495-ad96f14ebb21 service neutron] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:17.690390 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:17.776201 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971ce3b-6fa2-477a-b301-240052a73a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:17.777867 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971ce3b-6fa2-477a-b301-240052a73a50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:17.778450 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971ce3b-6fa2-477a-b301-240052a73a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:17.778983 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971ce3b-6fa2-477a-b301-240052a73a50 admin admin] PATH_INFO: `/v3/users/c151490d652140c198a85ac2c43f9506` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:17.781209 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5971ce3b-6fa2-477a-b301-240052a73a50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c151490d652140c198a85ac2c43f9506)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:17.785199 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5971ce3b-6fa2-477a-b301-240052a73a50 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:17.876814 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-5971ce3b-6fa2-477a-b301-240052a73a50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c151490d652140c198a85ac2c43f9506'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:17.878034 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-5971ce3b-6fa2-477a-b301-240052a73a50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c151490d652140c198a85ac2c43f9506'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:17.895895 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-5971ce3b-6fa2-477a-b301-240052a73a50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c151490d652140c198a85ac2c43f9506'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:17.897064 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-5971ce3b-6fa2-477a-b301-240052a73a50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c151490d652140c198a85ac2c43f9506'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:17.916077 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-5971ce3b-6fa2-477a-b301-240052a73a50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c151490d652140c198a85ac2c43f9506'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:17.960415 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1385/2761] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:46:17 2022] DELETE /identity/v3/users/c151490d652140c198a85ac2c43f9506 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:18.369916 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626a8035-2187-45f0-9349-ca5bd5cf1eb3 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:18.372269 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-626a8035-2187-45f0-9349-ca5bd5cf1eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:18.410630 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626a8035-2187-45f0-9349-ca5bd5cf1eb3 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:18.506531 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f510fce-4c67-4ca4-8bae-5e7e666ad5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:18.507380 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f510fce-4c67-4ca4-8bae-5e7e666ad5cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:18.507692 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f510fce-4c67-4ca4-8bae-5e7e666ad5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:18.507957 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f510fce-4c67-4ca4-8bae-5e7e666ad5cb admin admin] PATH_INFO: `/v3/projects/5bf2ef9f649542578bb77ac5dd4bd980` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:18.509722 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f510fce-4c67-4ca4-8bae-5e7e666ad5cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bf2ef9f649542578bb77ac5dd4bd980)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:18.512031 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f510fce-4c67-4ca4-8bae-5e7e666ad5cb admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:18.574485 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4f510fce-4c67-4ca4-8bae-5e7e666ad5cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bf2ef9f649542578bb77ac5dd4bd980', 'request_id': 'req-4f510fce-4c67-4ca4-8bae-5e7e666ad5cb'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:18.585428 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4f510fce-4c67-4ca4-8bae-5e7e666ad5cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bf2ef9f649542578bb77ac5dd4bd980', 'request_id': 'req-4f510fce-4c67-4ca4-8bae-5e7e666ad5cb'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:18.597306 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4f510fce-4c67-4ca4-8bae-5e7e666ad5cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bf2ef9f649542578bb77ac5dd4bd980', 'request_id': 'req-4f510fce-4c67-4ca4-8bae-5e7e666ad5cb'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:18.614852 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4f510fce-4c67-4ca4-8bae-5e7e666ad5cb admin admin] The token cache is being invalidate because project 5bf2ef9f649542578bb77ac5dd4bd980 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:46:18.615417 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4f510fce-4c67-4ca4-8bae-5e7e666ad5cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:18.616675 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4f510fce-4c67-4ca4-8bae-5e7e666ad5cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:18.618739 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1377/2762] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:46:18 2022] DELETE /identity/v3/projects/5bf2ef9f649542578bb77ac5dd4bd980 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:18.626808 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3af61-3150-43ce-9ac8-320a2c8e49ba None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:18.627150 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3af61-3150-43ce-9ac8-320a2c8e49ba None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:18.627475 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3af61-3150-43ce-9ac8-320a2c8e49ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:18.650236 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-5ac3af61-3150-43ce-9ac8-320a2c8e49ba None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:18.652366 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-5ac3af61-3150-43ce-9ac8-320a2c8e49ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:18.727121 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1386/2763] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:46:18.735886 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15582c-795e-4766-af7b-5b43311120ca None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:18.736428 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15582c-795e-4766-af7b-5b43311120ca None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:18.736924 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15582c-795e-4766-af7b-5b43311120ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:18.770864 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-fa15582c-795e-4766-af7b-5b43311120ca None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:18.772112 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-fa15582c-795e-4766-af7b-5b43311120ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:18.787819 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1378/2764] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:46:18.789474 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac3af61-3150-43ce-9ac8-320a2c8e49ba None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:18.800458 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac3af61-3150-43ce-9ac8-320a2c8e49ba None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:18.805603 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c3b1d-586d-4d82-963f-f7ef18ad67fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:18.806960 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c3b1d-586d-4d82-963f-f7ef18ad67fb admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:18.807244 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c3b1d-586d-4d82-963f-f7ef18ad67fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:18.807966 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c3b1d-586d-4d82-963f-f7ef18ad67fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:18.808258 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240c3b1d-586d-4d82-963f-f7ef18ad67fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:46:18.808572 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240c3b1d-586d-4d82-963f-f7ef18ad67fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:18.815614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240c3b1d-586d-4d82-963f-f7ef18ad67fb admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:18.824980 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1387/2765] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:46:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:46:18.830288 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa15582c-795e-4766-af7b-5b43311120ca None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:18.841327 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa15582c-795e-4766-af7b-5b43311120ca None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:18.846624 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23931dc4-4d92-4f4f-ad59-175c591c6e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:18.847316 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23931dc4-4d92-4f4f-ad59-175c591c6e93 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:18.847615 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23931dc4-4d92-4f4f-ad59-175c591c6e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:18.847875 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23931dc4-4d92-4f4f-ad59-175c591c6e93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:18.848767 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23931dc4-4d92-4f4f-ad59-175c591c6e93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:18.851005 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23931dc4-4d92-4f4f-ad59-175c591c6e93 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:18.886132 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1379/2766] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:46:18 2022] POST /identity/v3/projects => generated 387 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:46:18.887869 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c3b1d-586d-4d82-963f-f7ef18ad67fb admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:18.898909 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c3b1d-586d-4d82-963f-f7ef18ad67fb admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:18.911384 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:18.969865 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c4d525-8d8c-4f28-84f7-9fa1802af6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:18.976084 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4d525-8d8c-4f28-84f7-9fa1802af6b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:18.976902 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4d525-8d8c-4f28-84f7-9fa1802af6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:18.977503 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4d525-8d8c-4f28-84f7-9fa1802af6b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:18.978787 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c4d525-8d8c-4f28-84f7-9fa1802af6b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:18.981598 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c4d525-8d8c-4f28-84f7-9fa1802af6b3 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:19.024494 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1388/2767] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:46:18 2022] POST /identity/v3/users => generated 294 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:46:19.028754 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23931dc4-4d92-4f4f-ad59-175c591c6e93 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:19.040018 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23931dc4-4d92-4f4f-ad59-175c591c6e93 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:19.045857 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b29eb-baf4-4160-9211-ef2521b3cac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:19.046634 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b29eb-baf4-4160-9211-ef2521b3cac6 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:19.046925 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b29eb-baf4-4160-9211-ef2521b3cac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:19.047201 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b29eb-baf4-4160-9211-ef2521b3cac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:19.047933 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03b29eb-baf4-4160-9211-ef2521b3cac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:19.049914 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03b29eb-baf4-4160-9211-ef2521b3cac6 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:19.070830 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1380/2768] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:46:19 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:46:19.072248 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c4d525-8d8c-4f28-84f7-9fa1802af6b3 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:19.089678 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c4d525-8d8c-4f28-84f7-9fa1802af6b3 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:19.096649 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42643b06-0e0c-4b4a-8fa9-4f09c8daad51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:19.097682 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42643b06-0e0c-4b4a-8fa9-4f09c8daad51 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:19.098308 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42643b06-0e0c-4b4a-8fa9-4f09c8daad51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:19.098846 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42643b06-0e0c-4b4a-8fa9-4f09c8daad51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:19.099784 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42643b06-0e0c-4b4a-8fa9-4f09c8daad51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:19.101593 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42643b06-0e0c-4b4a-8fa9-4f09c8daad51 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:19.123724 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1389/2769] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:46:19 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:46:19.124938 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b29eb-baf4-4160-9211-ef2521b3cac6 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:19.138539 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b29eb-baf4-4160-9211-ef2521b3cac6 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:19.146743 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd42ec4-c6dd-47e8-ab74-6d8086fbdd9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:19.147679 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd42ec4-c6dd-47e8-ab74-6d8086fbdd9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:19.147966 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd42ec4-c6dd-47e8-ab74-6d8086fbdd9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:19.148243 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd42ec4-c6dd-47e8-ab74-6d8086fbdd9f admin admin] PATH_INFO: `/v3/projects/1816e1236a0b4121aa0d48c5fb378e94/users/8478b011c6db4080b5fee83d42a5eac6/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:19.173256 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd42ec4-c6dd-47e8-ab74-6d8086fbdd9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1816e1236a0b4121aa0d48c5fb378e94, user_id=8478b011c6db4080b5fee83d42a5eac6, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:19.176846 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd42ec4-c6dd-47e8-ab74-6d8086fbdd9f admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:19.218614 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1381/2770] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:46:19 2022] PUT /identity/v3/projects/1816e1236a0b4121aa0d48c5fb378e94/users/8478b011c6db4080b5fee83d42a5eac6/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:19.227388 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256404b0-48ef-42e0-9432-f92e26943a68 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:19.227733 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256404b0-48ef-42e0-9432-f92e26943a68 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:19.228548 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256404b0-48ef-42e0-9432-f92e26943a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:19.263437 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-256404b0-48ef-42e0-9432-f92e26943a68 None None] MFA Rules not processed for user `8478b011c6db4080b5fee83d42a5eac6`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:19.265698 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-256404b0-48ef-42e0-9432-f92e26943a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:19.407671 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1390/2771] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:46:19.413588 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd42ec4-c6dd-47e8-ab74-6d8086fbdd9f admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:19.421198 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:INSERT=4,keystone:DELETE=10,keystone:SELECT=86 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:19.526430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd42ec4-c6dd-47e8-ab74-6d8086fbdd9f admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:19.532777 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b7871-1e88-4350-b3a0-238474406c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:19.534151 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b7871-1e88-4350-b3a0-238474406c01 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:19.534459 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b7871-1e88-4350-b3a0-238474406c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:19.535246 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b7871-1e88-4350-b3a0-238474406c01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:19.536388 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19b7871-1e88-4350-b3a0-238474406c01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:19.539859 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19b7871-1e88-4350-b3a0-238474406c01 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:19.580844 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1382/2772] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:46:19 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:46:19.585400 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256404b0-48ef-42e0-9432-f92e26943a68 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:19.598117 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256404b0-48ef-42e0-9432-f92e26943a68 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:19.678674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d304a3c-43af-4eab-bd4f-c4a7bcb888d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:19.679371 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d304a3c-43af-4eab-bd4f-c4a7bcb888d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:19.679637 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d304a3c-43af-4eab-bd4f-c4a7bcb888d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:19.679948 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d304a3c-43af-4eab-bd4f-c4a7bcb888d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:19.680802 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d304a3c-43af-4eab-bd4f-c4a7bcb888d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:19.682961 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d304a3c-43af-4eab-bd4f-c4a7bcb888d2 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:19.711752 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1391/2773] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:46:19 2022] POST /identity/v3/users => generated 299 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:46:19.714236 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b7871-1e88-4350-b3a0-238474406c01 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:19.726031 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b7871-1e88-4350-b3a0-238474406c01 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:19.733385 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20915702-7337-4497-82c0-b0733769bbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:19.734647 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20915702-7337-4497-82c0-b0733769bbb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:19.735607 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20915702-7337-4497-82c0-b0733769bbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:19.735900 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20915702-7337-4497-82c0-b0733769bbb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:19.736682 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20915702-7337-4497-82c0-b0733769bbb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:19.738528 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20915702-7337-4497-82c0-b0733769bbb9 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:19.752337 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1383/2774] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:46:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:46:19.757191 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d304a3c-43af-4eab-bd4f-c4a7bcb888d2 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:19.775318 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d304a3c-43af-4eab-bd4f-c4a7bcb888d2 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:19.781522 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fef9d54-fc4f-4992-8975-b556093a27af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:19.782771 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef9d54-fc4f-4992-8975-b556093a27af admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:19.783081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef9d54-fc4f-4992-8975-b556093a27af admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:19.783337 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef9d54-fc4f-4992-8975-b556093a27af admin admin] PATH_INFO: `/v3/projects/db021b4f8ba94488859da29fce8f4c6f/users/ded31fb36d7c43669e1f99ba7b24f227/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:19.796156 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:19.814684 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fef9d54-fc4f-4992-8975-b556093a27af admin admin] RBAC: Authorizing `identity:create_grant(project_id=db021b4f8ba94488859da29fce8f4c6f, user_id=ded31fb36d7c43669e1f99ba7b24f227, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:19.819156 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fef9d54-fc4f-4992-8975-b556093a27af admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:19.844172 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1392/2775] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:46:19 2022] PUT /identity/v3/projects/db021b4f8ba94488859da29fce8f4c6f/users/ded31fb36d7c43669e1f99ba7b24f227/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:46:19.848644 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20915702-7337-4497-82c0-b0733769bbb9 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:19.868807 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20915702-7337-4497-82c0-b0733769bbb9 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:19.967560 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b44f2dd-0da5-40d2-b2bb-5ec3fbc91d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:19.968328 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44f2dd-0da5-40d2-b2bb-5ec3fbc91d5c admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:19.968616 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44f2dd-0da5-40d2-b2bb-5ec3fbc91d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:19.968904 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44f2dd-0da5-40d2-b2bb-5ec3fbc91d5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:19.969665 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b44f2dd-0da5-40d2-b2bb-5ec3fbc91d5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:19.971439 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b44f2dd-0da5-40d2-b2bb-5ec3fbc91d5c admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:19.987319 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1384/2776] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:46:19 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:46:19.992120 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fef9d54-fc4f-4992-8975-b556093a27af admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:20.004643 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fef9d54-fc4f-4992-8975-b556093a27af admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:20.010287 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c059b63-8e61-40ec-b53e-ebfec49538e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:20.011204 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c059b63-8e61-40ec-b53e-ebfec49538e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:20.011489 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c059b63-8e61-40ec-b53e-ebfec49538e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:20.011860 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c059b63-8e61-40ec-b53e-ebfec49538e3 admin admin] PATH_INFO: `/v3/projects/db021b4f8ba94488859da29fce8f4c6f/users/ded31fb36d7c43669e1f99ba7b24f227/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:20.014500 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c059b63-8e61-40ec-b53e-ebfec49538e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db021b4f8ba94488859da29fce8f4c6f, user_id=ded31fb36d7c43669e1f99ba7b24f227, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:20.017346 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c059b63-8e61-40ec-b53e-ebfec49538e3 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:20.033823 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1393/2777] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:46:19 2022] PUT /identity/v3/projects/db021b4f8ba94488859da29fce8f4c6f/users/ded31fb36d7c43669e1f99ba7b24f227/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:20.100860 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77066bc9-805d-4bb3-877d-a70c5d3b8e2e None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:20.101167 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77066bc9-805d-4bb3-877d-a70c5d3b8e2e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:20.101464 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77066bc9-805d-4bb3-877d-a70c5d3b8e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:20.134031 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-77066bc9-805d-4bb3-877d-a70c5d3b8e2e None None] MFA Rules not processed for user `ded31fb36d7c43669e1f99ba7b24f227`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:20.135910 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-77066bc9-805d-4bb3-877d-a70c5d3b8e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:20.243870 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:20.266274 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1385/2778] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:20 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:46:20.276698 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c059b63-8e61-40ec-b53e-ebfec49538e3 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:20.279715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-3c059b63-8e61-40ec-b53e-ebfec49538e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:20.427961 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c059b63-8e61-40ec-b53e-ebfec49538e3 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:20.435161 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f548e7-f2a4-477a-b244-cb0acc7d5ab9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:20.436690 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f548e7-f2a4-477a-b244-cb0acc7d5ab9 service cinder] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:20.437262 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f548e7-f2a4-477a-b244-cb0acc7d5ab9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:20.437614 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f548e7-f2a4-477a-b244-cb0acc7d5ab9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:20.459005 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f548e7-f2a4-477a-b244-cb0acc7d5ab9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:20.461500 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f548e7-f2a4-477a-b244-cb0acc7d5ab9 service cinder] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:20.476183 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1394/2779] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:46:20 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:46:20.526489 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77066bc9-805d-4bb3-877d-a70c5d3b8e2e None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:20.613949 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77066bc9-805d-4bb3-877d-a70c5d3b8e2e None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:20.620564 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485665a-9f72-443d-b655-d68fc15d40b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:20.621553 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485665a-9f72-443d-b655-d68fc15d40b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:20.621909 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485665a-9f72-443d-b655-d68fc15d40b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:20.622413 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485665a-9f72-443d-b655-d68fc15d40b3 admin admin] PATH_INFO: `/v3/users/8478b011c6db4080b5fee83d42a5eac6` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:20.624342 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2485665a-9f72-443d-b655-d68fc15d40b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8478b011c6db4080b5fee83d42a5eac6)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:20.625924 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2485665a-9f72-443d-b655-d68fc15d40b3 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:20.700403 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2485665a-9f72-443d-b655-d68fc15d40b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8478b011c6db4080b5fee83d42a5eac6'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:20.702746 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2485665a-9f72-443d-b655-d68fc15d40b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8478b011c6db4080b5fee83d42a5eac6'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:20.702980 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2485665a-9f72-443d-b655-d68fc15d40b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8478b011c6db4080b5fee83d42a5eac6'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:20.721004 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2485665a-9f72-443d-b655-d68fc15d40b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8478b011c6db4080b5fee83d42a5eac6'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:20.734805 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2485665a-9f72-443d-b655-d68fc15d40b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8478b011c6db4080b5fee83d42a5eac6'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:20.762347 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1386/2780] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:46:20 2022] DELETE /identity/v3/users/8478b011c6db4080b5fee83d42a5eac6 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:20.764663 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2848b66-5261-4fbb-930f-fc5375849995 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:20.765157 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2848b66-5261-4fbb-930f-fc5375849995 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:20.765268 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2848b66-5261-4fbb-930f-fc5375849995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:20.788923 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-d2848b66-5261-4fbb-930f-fc5375849995 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:20.791480 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-d2848b66-5261-4fbb-930f-fc5375849995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:20.893560 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1395/2781] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:46:20.900593 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485665a-9f72-443d-b655-d68fc15d40b3 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:20.902574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-2485665a-9f72-443d-b655-d68fc15d40b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:20.989702 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485665a-9f72-443d-b655-d68fc15d40b3 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:20.994963 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bad62e5-b0d7-480e-95f9-389f2475fd1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:20.995760 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad62e5-b0d7-480e-95f9-389f2475fd1e service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:20.996266 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad62e5-b0d7-480e-95f9-389f2475fd1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:20.996685 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad62e5-b0d7-480e-95f9-389f2475fd1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:21.009816 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bad62e5-b0d7-480e-95f9-389f2475fd1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:21.011892 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bad62e5-b0d7-480e-95f9-389f2475fd1e service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:21.025156 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1387/2782] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:46:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:46:21.562509 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2848b66-5261-4fbb-930f-fc5375849995 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:21.564131 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-d2848b66-5261-4fbb-930f-fc5375849995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:21.588115 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2848b66-5261-4fbb-930f-fc5375849995 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:21.594402 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a779b3-7889-44f3-86f0-2f18c28823bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:21.595293 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a779b3-7889-44f3-86f0-2f18c28823bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:21.595578 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a779b3-7889-44f3-86f0-2f18c28823bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:21.595850 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a779b3-7889-44f3-86f0-2f18c28823bd admin admin] PATH_INFO: `/v3/projects/1816e1236a0b4121aa0d48c5fb378e94` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:21.599671 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a779b3-7889-44f3-86f0-2f18c28823bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=1816e1236a0b4121aa0d48c5fb378e94)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:21.601827 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a779b3-7889-44f3-86f0-2f18c28823bd admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:21.667785 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-c9a779b3-7889-44f3-86f0-2f18c28823bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1816e1236a0b4121aa0d48c5fb378e94', 'request_id': 'req-c9a779b3-7889-44f3-86f0-2f18c28823bd'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:21.677039 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-c9a779b3-7889-44f3-86f0-2f18c28823bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1816e1236a0b4121aa0d48c5fb378e94', 'request_id': 'req-c9a779b3-7889-44f3-86f0-2f18c28823bd'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:21.686386 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-c9a779b3-7889-44f3-86f0-2f18c28823bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1816e1236a0b4121aa0d48c5fb378e94', 'request_id': 'req-c9a779b3-7889-44f3-86f0-2f18c28823bd'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:21.695693 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:21.701572 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-c9a779b3-7889-44f3-86f0-2f18c28823bd admin admin] The token cache is being invalidate because project 1816e1236a0b4121aa0d48c5fb378e94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:46:21.703277 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-c9a779b3-7889-44f3-86f0-2f18c28823bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:21.704669 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-c9a779b3-7889-44f3-86f0-2f18c28823bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:21.707168 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1396/2783] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:46:21 2022] DELETE /identity/v3/projects/1816e1236a0b4121aa0d48c5fb378e94 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:21.711370 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bad62e5-b0d7-480e-95f9-389f2475fd1e service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:21.727197 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bad62e5-b0d7-480e-95f9-389f2475fd1e service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:21.750159 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:21.810991 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60d59c-74a3-41fc-ac3f-cf499a6ea1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:21.812345 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60d59c-74a3-41fc-ac3f-cf499a6ea1e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:21.812873 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60d59c-74a3-41fc-ac3f-cf499a6ea1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:21.813365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60d59c-74a3-41fc-ac3f-cf499a6ea1e8 admin admin] PATH_INFO: `/v3/users/ded31fb36d7c43669e1f99ba7b24f227` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:21.814914 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60d59c-74a3-41fc-ac3f-cf499a6ea1e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ded31fb36d7c43669e1f99ba7b24f227)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:21.817174 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60d59c-74a3-41fc-ac3f-cf499a6ea1e8 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:21.901179 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-cd60d59c-74a3-41fc-ac3f-cf499a6ea1e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ded31fb36d7c43669e1f99ba7b24f227'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:21.902081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-cd60d59c-74a3-41fc-ac3f-cf499a6ea1e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ded31fb36d7c43669e1f99ba7b24f227'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:21.902777 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-cd60d59c-74a3-41fc-ac3f-cf499a6ea1e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ded31fb36d7c43669e1f99ba7b24f227'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:21.918289 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-cd60d59c-74a3-41fc-ac3f-cf499a6ea1e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ded31fb36d7c43669e1f99ba7b24f227'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:21.931864 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-cd60d59c-74a3-41fc-ac3f-cf499a6ea1e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ded31fb36d7c43669e1f99ba7b24f227'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:21.949067 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1388/2784] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:46:21 2022] DELETE /identity/v3/users/ded31fb36d7c43669e1f99ba7b24f227 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:22.560190 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a779b3-7889-44f3-86f0-2f18c28823bd admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:22.562475 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-c9a779b3-7889-44f3-86f0-2f18c28823bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:22.594814 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a779b3-7889-44f3-86f0-2f18c28823bd admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:22.677898 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7764ad1a-8d3b-44c3-b2c0-38ba0838447d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:22.679025 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764ad1a-8d3b-44c3-b2c0-38ba0838447d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:22.679364 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764ad1a-8d3b-44c3-b2c0-38ba0838447d admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:22.679646 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764ad1a-8d3b-44c3-b2c0-38ba0838447d admin admin] PATH_INFO: `/v3/projects/db021b4f8ba94488859da29fce8f4c6f` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:22.683906 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7764ad1a-8d3b-44c3-b2c0-38ba0838447d admin admin] RBAC: Authorizing `identity:delete_project(project_id=db021b4f8ba94488859da29fce8f4c6f)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:22.686939 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7764ad1a-8d3b-44c3-b2c0-38ba0838447d admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:22.754135 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-7764ad1a-8d3b-44c3-b2c0-38ba0838447d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db021b4f8ba94488859da29fce8f4c6f', 'request_id': 'req-7764ad1a-8d3b-44c3-b2c0-38ba0838447d'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:22.791182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-7764ad1a-8d3b-44c3-b2c0-38ba0838447d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db021b4f8ba94488859da29fce8f4c6f', 'request_id': 'req-7764ad1a-8d3b-44c3-b2c0-38ba0838447d'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:22.806361 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-7764ad1a-8d3b-44c3-b2c0-38ba0838447d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db021b4f8ba94488859da29fce8f4c6f', 'request_id': 'req-7764ad1a-8d3b-44c3-b2c0-38ba0838447d'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:22.843460 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-7764ad1a-8d3b-44c3-b2c0-38ba0838447d admin admin] The token cache is being invalidate because project db021b4f8ba94488859da29fce8f4c6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:46:22.844058 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-7764ad1a-8d3b-44c3-b2c0-38ba0838447d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:22.844941 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-7764ad1a-8d3b-44c3-b2c0-38ba0838447d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:22.850141 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1397/2785] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:46:22 2022] DELETE /identity/v3/projects/db021b4f8ba94488859da29fce8f4c6f => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:22.855022 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f57ca6-8c6a-43d6-a9d2-131e4e80fb18 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:22.855779 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f57ca6-8c6a-43d6-a9d2-131e4e80fb18 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:22.856437 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f57ca6-8c6a-43d6-a9d2-131e4e80fb18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:22.945010 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-28f57ca6-8c6a-43d6-a9d2-131e4e80fb18 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:22.947264 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-28f57ca6-8c6a-43d6-a9d2-131e4e80fb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:23.073346 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1389/2786] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:46:23.075440 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3f2b6-55b3-4b1d-83dd-5cc3bed292a6 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:23.075888 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3f2b6-55b3-4b1d-83dd-5cc3bed292a6 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:23.076304 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3f2b6-55b3-4b1d-83dd-5cc3bed292a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:23.105031 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-86b3f2b6-55b3-4b1d-83dd-5cc3bed292a6 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:23.106344 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-86b3f2b6-55b3-4b1d-83dd-5cc3bed292a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:23.118921 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1398/2787] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:46:23.120180 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f57ca6-8c6a-43d6-a9d2-131e4e80fb18 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:23.130881 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f57ca6-8c6a-43d6-a9d2-131e4e80fb18 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:23.136905 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e97af-2c60-464c-a696-257f39075be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:23.137637 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e97af-2c60-464c-a696-257f39075be5 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:23.137927 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e97af-2c60-464c-a696-257f39075be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:23.138257 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e97af-2c60-464c-a696-257f39075be5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:23.139014 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8e97af-2c60-464c-a696-257f39075be5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:46:23.139326 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8e97af-2c60-464c-a696-257f39075be5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:23.140977 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8e97af-2c60-464c-a696-257f39075be5 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:23.157809 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1390/2788] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:46:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:46:23.159430 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3f2b6-55b3-4b1d-83dd-5cc3bed292a6 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:23.173364 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3f2b6-55b3-4b1d-83dd-5cc3bed292a6 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:23.179682 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a61e6-7fbc-4f62-8be7-fd06081258f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:23.180355 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a61e6-7fbc-4f62-8be7-fd06081258f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:23.180636 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a61e6-7fbc-4f62-8be7-fd06081258f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:23.181455 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a61e6-7fbc-4f62-8be7-fd06081258f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:23.182580 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092a61e6-7fbc-4f62-8be7-fd06081258f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:23.184106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092a61e6-7fbc-4f62-8be7-fd06081258f7 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:23.225680 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1399/2789] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:46:23 2022] POST /identity/v3/projects => generated 391 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:46:23.230030 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e97af-2c60-464c-a696-257f39075be5 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:23.246829 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e97af-2c60-464c-a696-257f39075be5 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:23.285079 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:23.350569 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3902ac5-ddef-4fec-9aea-22f4c103d208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:23.351412 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3902ac5-ddef-4fec-9aea-22f4c103d208 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:23.351758 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3902ac5-ddef-4fec-9aea-22f4c103d208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:23.352107 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3902ac5-ddef-4fec-9aea-22f4c103d208 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:23.352950 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3902ac5-ddef-4fec-9aea-22f4c103d208 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:23.355268 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3902ac5-ddef-4fec-9aea-22f4c103d208 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:23.399267 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1391/2790] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:46:23 2022] POST /identity/v3/users => generated 296 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:46:23.403481 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a61e6-7fbc-4f62-8be7-fd06081258f7 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:23.427634 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a61e6-7fbc-4f62-8be7-fd06081258f7 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:23.443113 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063abc0b-ce54-471f-b3ec-24fb5338e67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:23.444262 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063abc0b-ce54-471f-b3ec-24fb5338e67a admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:23.444755 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063abc0b-ce54-471f-b3ec-24fb5338e67a admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:23.445299 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063abc0b-ce54-471f-b3ec-24fb5338e67a admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:23.446344 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063abc0b-ce54-471f-b3ec-24fb5338e67a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:23.454761 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063abc0b-ce54-471f-b3ec-24fb5338e67a admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:23.469863 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1400/2791] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:46:23 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:46:23.474910 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3902ac5-ddef-4fec-9aea-22f4c103d208 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:23.492317 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3902ac5-ddef-4fec-9aea-22f4c103d208 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:23.500119 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd51bd-35d7-4c04-a599-8e1967fc2eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:23.500946 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd51bd-35d7-4c04-a599-8e1967fc2eaa admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:23.501389 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd51bd-35d7-4c04-a599-8e1967fc2eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:23.501760 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd51bd-35d7-4c04-a599-8e1967fc2eaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:23.502650 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cd51bd-35d7-4c04-a599-8e1967fc2eaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:23.504421 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cd51bd-35d7-4c04-a599-8e1967fc2eaa admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:23.524436 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1392/2792] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:46:23 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:46:23.529231 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063abc0b-ce54-471f-b3ec-24fb5338e67a admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:23.539766 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063abc0b-ce54-471f-b3ec-24fb5338e67a admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:23.545184 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db639437-6ae8-477c-815b-5c32ee7960d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:23.546182 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db639437-6ae8-477c-815b-5c32ee7960d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:23.546539 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db639437-6ae8-477c-815b-5c32ee7960d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:23.546883 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db639437-6ae8-477c-815b-5c32ee7960d4 admin admin] PATH_INFO: `/v3/projects/46f596955a244cb69437b9ccc5668239/users/d3f449902fcf47b4b61158cda7c42eaf/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:23.574072 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db639437-6ae8-477c-815b-5c32ee7960d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46f596955a244cb69437b9ccc5668239, user_id=d3f449902fcf47b4b61158cda7c42eaf, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:23.576904 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db639437-6ae8-477c-815b-5c32ee7960d4 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:23.610217 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1401/2793] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:46:23 2022] PUT /identity/v3/projects/46f596955a244cb69437b9ccc5668239/users/d3f449902fcf47b4b61158cda7c42eaf/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:23.615936 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e8744-7eae-46ab-accb-2510fe1ea060 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:23.617532 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e8744-7eae-46ab-accb-2510fe1ea060 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:23.620399 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e8744-7eae-46ab-accb-2510fe1ea060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:23.663581 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfeabd0-f48c-453d-bbe7-05f48f4c87c5 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:23.663880 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfeabd0-f48c-453d-bbe7-05f48f4c87c5 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:23.664170 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfeabd0-f48c-453d-bbe7-05f48f4c87c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:23.671747 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-642e8744-7eae-46ab-accb-2510fe1ea060 None None] MFA Rules not processed for user `d3f449902fcf47b4b61158cda7c42eaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:23.672898 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-642e8744-7eae-46ab-accb-2510fe1ea060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:23.690368 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-3bfeabd0-f48c-453d-bbe7-05f48f4c87c5 None None] MFA Rules not processed for user `67aa048d409c46e9a729aec9d1b2286e`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:23.691613 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-3bfeabd0-f48c-453d-bbe7-05f48f4c87c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:23.724530 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:23.768542 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1393/2794] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:46:23.774764 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642e8744-7eae-46ab-accb-2510fe1ea060 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:23.779245 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-642e8744-7eae-46ab-accb-2510fe1ea060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:23.783700 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1402/2795] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Jun 15 19:46:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:46:23.806306 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfeabd0-f48c-453d-bbe7-05f48f4c87c5 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:23.808772 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-3bfeabd0-f48c-453d-bbe7-05f48f4c87c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:23.879785 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642e8744-7eae-46ab-accb-2510fe1ea060 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:23.886565 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f92a75-be19-4769-8354-978f658431af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5882def40a4e0d8a792eecc12440a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:23.887259 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f92a75-be19-4769-8354-978f658431af service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:23.887544 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f92a75-be19-4769-8354-978f658431af service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:23.887880 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f92a75-be19-4769-8354-978f658431af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:23.909560 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f92a75-be19-4769-8354-978f658431af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:23.911640 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f92a75-be19-4769-8354-978f658431af service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:23.938491 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1394/2796] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:46:23 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:46:23.958753 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfeabd0-f48c-453d-bbe7-05f48f4c87c5 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:23.965973 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef80d4-4325-4078-ab3a-3d800de0762d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176d5775a70f4524a9b7fe1ce9d426ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:23.966716 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef80d4-4325-4078-ab3a-3d800de0762d service swift] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:23.966972 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef80d4-4325-4078-ab3a-3d800de0762d service swift] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:23.967253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef80d4-4325-4078-ab3a-3d800de0762d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:23.981987 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ef80d4-4325-4078-ab3a-3d800de0762d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:23.983654 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ef80d4-4325-4078-ab3a-3d800de0762d service swift] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:24.001532 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1403/2797] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Jun 15 19:46:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 15 19:46:24.763577 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac517b-83b5-46ae-acc6-b138f190b0c2 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:24.763867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac517b-83b5-46ae-acc6-b138f190b0c2 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:24.764148 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac517b-83b5-46ae-acc6-b138f190b0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:24.786310 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-a4ac517b-83b5-46ae-acc6-b138f190b0c2 None None] MFA Rules not processed for user `67aa048d409c46e9a729aec9d1b2286e`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:24.787960 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a4ac517b-83b5-46ae-acc6-b138f190b0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:24.796347 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1395/2798] 10.222.0.15 () {58 vars in 967 bytes} [Wed Jun 15 19:46:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:46:24.812001 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef80d4-4325-4078-ab3a-3d800de0762d service swift] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:24.824139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef80d4-4325-4078-ab3a-3d800de0762d service swift] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:24.831205 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9695834-151a-442b-b0a8-ea0b1e9fa68a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176d5775a70f4524a9b7fe1ce9d426ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:24.832270 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9695834-151a-442b-b0a8-ea0b1e9fa68a service swift] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:24.832715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9695834-151a-442b-b0a8-ea0b1e9fa68a service swift] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:24.832916 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9695834-151a-442b-b0a8-ea0b1e9fa68a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:24.844023 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9695834-151a-442b-b0a8-ea0b1e9fa68a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:24.845731 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9695834-151a-442b-b0a8-ea0b1e9fa68a service swift] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:24.856592 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1404/2799] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Jun 15 19:46:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 15 19:46:25.118423 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22c0ef-a438-4cb8-a11d-eef13b6b46a5 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:25.119322 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22c0ef-a438-4cb8-a11d-eef13b6b46a5 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:25.119751 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22c0ef-a438-4cb8-a11d-eef13b6b46a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:25.140945 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-8d22c0ef-a438-4cb8-a11d-eef13b6b46a5 None None] MFA Rules not processed for user `67aa048d409c46e9a729aec9d1b2286e`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:25.142199 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8d22c0ef-a438-4cb8-a11d-eef13b6b46a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:25.150652 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1396/2800] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Jun 15 19:46:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:46:25.171088 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9695834-151a-442b-b0a8-ea0b1e9fa68a service swift] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:25.179797 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9695834-151a-442b-b0a8-ea0b1e9fa68a service swift] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:25.190794 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f82d7b-7678-44f7-b361-406c6ffc589e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176d5775a70f4524a9b7fe1ce9d426ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:25.191469 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f82d7b-7678-44f7-b361-406c6ffc589e service swift] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:25.191753 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f82d7b-7678-44f7-b361-406c6ffc589e service swift] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:25.192061 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f82d7b-7678-44f7-b361-406c6ffc589e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:25.202558 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f82d7b-7678-44f7-b361-406c6ffc589e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:25.204171 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f82d7b-7678-44f7-b361-406c6ffc589e service swift] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:25.226149 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1405/2801] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Jun 15 19:46:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 19:46:25.878881 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa3083-1a83-49b6-bf5a-aaacfd3aa199 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:25.881705 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa3083-1a83-49b6-bf5a-aaacfd3aa199 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:25.882116 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa3083-1a83-49b6-bf5a-aaacfd3aa199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:25.907212 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-a5aa3083-1a83-49b6-bf5a-aaacfd3aa199 None None] MFA Rules not processed for user `67aa048d409c46e9a729aec9d1b2286e`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:25.908809 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a5aa3083-1a83-49b6-bf5a-aaacfd3aa199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:25.918738 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1397/2802] 10.222.0.15 () {58 vars in 967 bytes} [Wed Jun 15 19:46:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:46:25.939749 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f82d7b-7678-44f7-b361-406c6ffc589e service swift] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:25.952309 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f82d7b-7678-44f7-b361-406c6ffc589e service swift] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:25.960874 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00f8cd-e222-41e0-9605-5b7a111a8578 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176d5775a70f4524a9b7fe1ce9d426ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:25.961766 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00f8cd-e222-41e0-9605-5b7a111a8578 service swift] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:25.962215 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00f8cd-e222-41e0-9605-5b7a111a8578 service swift] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:25.962662 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00f8cd-e222-41e0-9605-5b7a111a8578 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:25.972371 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d00f8cd-e222-41e0-9605-5b7a111a8578 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:25.976594 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d00f8cd-e222-41e0-9605-5b7a111a8578 service swift] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:25.987967 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1406/2803] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Jun 15 19:46:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 19:46:29.180111 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa3083-1a83-49b6-bf5a-aaacfd3aa199 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:29.181742 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a5aa3083-1a83-49b6-bf5a-aaacfd3aa199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:29.203592 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:29.249633 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa3083-1a83-49b6-bf5a-aaacfd3aa199 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:29.254138 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5aa6f-d860-42cd-ae8f-46f70fc94cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:29.254997 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5aa6f-d860-42cd-ae8f-46f70fc94cd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:29.255276 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5aa6f-d860-42cd-ae8f-46f70fc94cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:29.255546 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5aa6f-d860-42cd-ae8f-46f70fc94cd6 admin admin] PATH_INFO: `/v3/users/b65a1f14f9bc46c59214ed240316aa86` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:29.256754 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a5aa6f-d860-42cd-ae8f-46f70fc94cd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b65a1f14f9bc46c59214ed240316aa86)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:29.258762 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a5aa6f-d860-42cd-ae8f-46f70fc94cd6 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:29.310744 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-c5a5aa6f-d860-42cd-ae8f-46f70fc94cd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b65a1f14f9bc46c59214ed240316aa86'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:29.311398 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-c5a5aa6f-d860-42cd-ae8f-46f70fc94cd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b65a1f14f9bc46c59214ed240316aa86'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:29.312004 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-c5a5aa6f-d860-42cd-ae8f-46f70fc94cd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b65a1f14f9bc46c59214ed240316aa86'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:29.323734 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-c5a5aa6f-d860-42cd-ae8f-46f70fc94cd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b65a1f14f9bc46c59214ed240316aa86'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:29.330728 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-c5a5aa6f-d860-42cd-ae8f-46f70fc94cd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b65a1f14f9bc46c59214ed240316aa86'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:29.344743 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1398/2804] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:46:29 2022] DELETE /identity/v3/users/b65a1f14f9bc46c59214ed240316aa86 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:29.349277 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2101b-4f20-42e9-b0a5-6bd2e010e94f None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:29.349715 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2101b-4f20-42e9-b0a5-6bd2e010e94f None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:29.349920 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2101b-4f20-42e9-b0a5-6bd2e010e94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:29.368302 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-35b2101b-4f20-42e9-b0a5-6bd2e010e94f None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:29.369551 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-35b2101b-4f20-42e9-b0a5-6bd2e010e94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:29.430508 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1407/2805] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:46:29.440195 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5aa6f-d860-42cd-ae8f-46f70fc94cd6 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:29.441658 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c5a5aa6f-d860-42cd-ae8f-46f70fc94cd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:29.500296 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5aa6f-d860-42cd-ae8f-46f70fc94cd6 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:29.503999 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d07c1f-fdd2-4013-9f27-cb99a6d37568 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:29.504654 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d07c1f-fdd2-4013-9f27-cb99a6d37568 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:29.504963 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d07c1f-fdd2-4013-9f27-cb99a6d37568 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:29.505223 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d07c1f-fdd2-4013-9f27-cb99a6d37568 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:29.515085 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d07c1f-fdd2-4013-9f27-cb99a6d37568 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:29.516596 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d07c1f-fdd2-4013-9f27-cb99a6d37568 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:29.527010 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1399/2806] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:46:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:46:29.699507 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b2101b-4f20-42e9-b0a5-6bd2e010e94f None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:29.700950 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-35b2101b-4f20-42e9-b0a5-6bd2e010e94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:29.713738 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b2101b-4f20-42e9-b0a5-6bd2e010e94f None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:29.718668 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7389fdb-3a3a-4d3f-b47a-d13b71a91adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:29.719566 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7389fdb-3a3a-4d3f-b47a-d13b71a91adf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:29.720062 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7389fdb-3a3a-4d3f-b47a-d13b71a91adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:29.720397 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7389fdb-3a3a-4d3f-b47a-d13b71a91adf admin admin] PATH_INFO: `/v3/projects/64a686a0b5d04802981b602dceb98804` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:29.721353 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7389fdb-3a3a-4d3f-b47a-d13b71a91adf admin admin] RBAC: Authorizing `identity:delete_project(project_id=64a686a0b5d04802981b602dceb98804)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:29.723370 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7389fdb-3a3a-4d3f-b47a-d13b71a91adf admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:29.745189 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:29.775880 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-d7389fdb-3a3a-4d3f-b47a-d13b71a91adf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64a686a0b5d04802981b602dceb98804', 'request_id': 'req-d7389fdb-3a3a-4d3f-b47a-d13b71a91adf'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:29.782970 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-d7389fdb-3a3a-4d3f-b47a-d13b71a91adf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64a686a0b5d04802981b602dceb98804', 'request_id': 'req-d7389fdb-3a3a-4d3f-b47a-d13b71a91adf'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:29.789289 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-d7389fdb-3a3a-4d3f-b47a-d13b71a91adf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64a686a0b5d04802981b602dceb98804', 'request_id': 'req-d7389fdb-3a3a-4d3f-b47a-d13b71a91adf'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:29.801978 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-d7389fdb-3a3a-4d3f-b47a-d13b71a91adf admin admin] The token cache is being invalidate because project 64a686a0b5d04802981b602dceb98804 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:46:29.802251 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-d7389fdb-3a3a-4d3f-b47a-d13b71a91adf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:29.803528 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-d7389fdb-3a3a-4d3f-b47a-d13b71a91adf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:29.806219 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1408/2807] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:46:29 2022] DELETE /identity/v3/projects/64a686a0b5d04802981b602dceb98804 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:29.812771 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536efe8-75de-441c-9ab4-044616557aee None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:29.813066 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536efe8-75de-441c-9ab4-044616557aee None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:29.813344 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536efe8-75de-441c-9ab4-044616557aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:29.840900 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-3536efe8-75de-441c-9ab4-044616557aee None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:29.842342 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3536efe8-75de-441c-9ab4-044616557aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:29.902642 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:29.919369 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc37f3a-780f-416e-8b9e-550f999d3d4f None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:29.919691 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc37f3a-780f-416e-8b9e-550f999d3d4f None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:29.919976 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc37f3a-780f-416e-8b9e-550f999d3d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:29.935694 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1400/2808] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:46:29.943515 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212904ed-218b-497a-a9fc-b7580301dddd None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:29.943811 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212904ed-218b-497a-a9fc-b7580301dddd None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:29.944161 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212904ed-218b-497a-a9fc-b7580301dddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:29.946249 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-ecc37f3a-780f-416e-8b9e-550f999d3d4f None None] MFA Rules not processed for user `67aa048d409c46e9a729aec9d1b2286e`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:29.948033 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-ecc37f3a-780f-416e-8b9e-550f999d3d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:29.964643 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-212904ed-218b-497a-a9fc-b7580301dddd None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:29.965926 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-212904ed-218b-497a-a9fc-b7580301dddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:29.976463 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1401/2809] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:46:29.980600 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212904ed-218b-497a-a9fc-b7580301dddd None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:30.002079 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212904ed-218b-497a-a9fc-b7580301dddd None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:30.007464 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476ede4-7ea1-47d1-ab4f-24ff64078bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:30.008179 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476ede4-7ea1-47d1-ab4f-24ff64078bb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:30.008468 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476ede4-7ea1-47d1-ab4f-24ff64078bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:30.009872 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476ede4-7ea1-47d1-ab4f-24ff64078bb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:30.010563 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a476ede4-7ea1-47d1-ab4f-24ff64078bb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:46:30.010870 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a476ede4-7ea1-47d1-ab4f-24ff64078bb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:30.012635 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a476ede4-7ea1-47d1-ab4f-24ff64078bb9 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:30.028131 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1402/2810] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:46:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 19:46:30.028350 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1409/2811] 10.222.0.15 () {58 vars in 967 bytes} [Wed Jun 15 19:46:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:46:30.029221 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476ede4-7ea1-47d1-ab4f-24ff64078bb9 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:30.039711 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476ede4-7ea1-47d1-ab4f-24ff64078bb9 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:30.040661 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc37f3a-780f-416e-8b9e-550f999d3d4f None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:30.042082 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-ecc37f3a-780f-416e-8b9e-550f999d3d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:30.045640 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dd0a60-f960-4cfe-bb10-96b9e7aec8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:30.047359 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd0a60-f960-4cfe-bb10-96b9e7aec8ff admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:30.047359 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd0a60-f960-4cfe-bb10-96b9e7aec8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:30.047635 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd0a60-f960-4cfe-bb10-96b9e7aec8ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:30.048520 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dd0a60-f960-4cfe-bb10-96b9e7aec8ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:30.050748 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dd0a60-f960-4cfe-bb10-96b9e7aec8ff admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:30.078910 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1403/2812] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:46:30 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:46:30.082938 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dd0a60-f960-4cfe-bb10-96b9e7aec8ff admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:30.114477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc37f3a-780f-416e-8b9e-550f999d3d4f None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:30.145295 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dd0a60-f960-4cfe-bb10-96b9e7aec8ff admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:30.149969 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac00336-f881-41ca-b5b8-77f6b2adbd48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:30.150688 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac00336-f881-41ca-b5b8-77f6b2adbd48 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:30.150974 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac00336-f881-41ca-b5b8-77f6b2adbd48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:30.151251 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac00336-f881-41ca-b5b8-77f6b2adbd48 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:30.151979 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac00336-f881-41ca-b5b8-77f6b2adbd48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:30.153890 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac00336-f881-41ca-b5b8-77f6b2adbd48 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:30.163874 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:30.175678 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1404/2813] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:46:30 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:46:30.179569 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b124b-9e68-4f0a-9df4-f8f27fa2d046 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176d5775a70f4524a9b7fe1ce9d426ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:30.179731 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac00336-f881-41ca-b5b8-77f6b2adbd48 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:30.180240 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b124b-9e68-4f0a-9df4-f8f27fa2d046 service swift] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:30.181057 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b124b-9e68-4f0a-9df4-f8f27fa2d046 service swift] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:30.181343 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b124b-9e68-4f0a-9df4-f8f27fa2d046 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:30.190637 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac00336-f881-41ca-b5b8-77f6b2adbd48 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:30.195621 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16acbb2-038c-4d9b-a142-7476f9192ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:30.196340 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16acbb2-038c-4d9b-a142-7476f9192ac9 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:30.196562 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16acbb2-038c-4d9b-a142-7476f9192ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:30.196842 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16acbb2-038c-4d9b-a142-7476f9192ac9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:30.198106 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16acbb2-038c-4d9b-a142-7476f9192ac9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:30.199777 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16acbb2-038c-4d9b-a142-7476f9192ac9 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:30.211442 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1405/2814] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:46:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:46:30.215704 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16acbb2-038c-4d9b-a142-7476f9192ac9 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:30.226755 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16acbb2-038c-4d9b-a142-7476f9192ac9 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:30.232934 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb5c10c-ea6e-4e67-8820-c5a3a1dee604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:30.233612 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5c10c-ea6e-4e67-8820-c5a3a1dee604 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:30.233896 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5c10c-ea6e-4e67-8820-c5a3a1dee604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:30.234217 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5c10c-ea6e-4e67-8820-c5a3a1dee604 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:30.234971 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb5c10c-ea6e-4e67-8820-c5a3a1dee604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:30.236619 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb5c10c-ea6e-4e67-8820-c5a3a1dee604 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:30.249472 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138b124b-9e68-4f0a-9df4-f8f27fa2d046 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:30.250904 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1406/2815] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:46:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:46:30.254117 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138b124b-9e68-4f0a-9df4-f8f27fa2d046 service swift] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:30.255074 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb5c10c-ea6e-4e67-8820-c5a3a1dee604 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:30.261724 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1410/2816] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Jun 15 19:46:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 15 19:46:30.267935 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb5c10c-ea6e-4e67-8820-c5a3a1dee604 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:30.273300 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f99cd-06e1-4694-a9a4-f507ff1246ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:30.274454 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f99cd-06e1-4694-a9a4-f507ff1246ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:30.274946 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f99cd-06e1-4694-a9a4-f507ff1246ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:30.275243 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f99cd-06e1-4694-a9a4-f507ff1246ac admin admin] PATH_INFO: `/v3/projects/b7cdb8d3ddc84d0fbd6128c3b46d05f7/users/f3eb282e55c941ef9fefb57e14a708f1/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:30.293782 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f99cd-06e1-4694-a9a4-f507ff1246ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7cdb8d3ddc84d0fbd6128c3b46d05f7, user_id=f3eb282e55c941ef9fefb57e14a708f1, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:30.296529 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f99cd-06e1-4694-a9a4-f507ff1246ac admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:30.312563 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1407/2817] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:46:30 2022] PUT /identity/v3/projects/b7cdb8d3ddc84d0fbd6128c3b46d05f7/users/f3eb282e55c941ef9fefb57e14a708f1/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:46:30.317208 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05083372-3b25-4a12-b499-28b7eb06d4a6 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:30.317504 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05083372-3b25-4a12-b499-28b7eb06d4a6 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:30.317786 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05083372-3b25-4a12-b499-28b7eb06d4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:30.338345 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-05083372-3b25-4a12-b499-28b7eb06d4a6 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:30.339599 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-05083372-3b25-4a12-b499-28b7eb06d4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:30.408971 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1411/2818] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:46:30.422117 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f99cd-06e1-4694-a9a4-f507ff1246ac admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:30.424297 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-665f99cd-06e1-4694-a9a4-f507ff1246ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:30.493654 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:30.506959 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f99cd-06e1-4694-a9a4-f507ff1246ac admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:30.511336 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d599a7-3e1e-40cf-9a3b-ab980fd5c72e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:30.512153 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d599a7-3e1e-40cf-9a3b-ab980fd5c72e service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:30.512616 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d599a7-3e1e-40cf-9a3b-ab980fd5c72e service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:30.512616 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d599a7-3e1e-40cf-9a3b-ab980fd5c72e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:30.524096 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d599a7-3e1e-40cf-9a3b-ab980fd5c72e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:30.526152 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d599a7-3e1e-40cf-9a3b-ab980fd5c72e service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:30.541276 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1408/2819] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:46:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:46:34.712405 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05083372-3b25-4a12-b499-28b7eb06d4a6 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:34.717955 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-05083372-3b25-4a12-b499-28b7eb06d4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:34.736923 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05083372-3b25-4a12-b499-28b7eb06d4a6 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:34.745020 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220443be-ad12-4a13-8ec7-897e9cb884b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:34.746572 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220443be-ad12-4a13-8ec7-897e9cb884b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:34.747075 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220443be-ad12-4a13-8ec7-897e9cb884b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:34.747572 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220443be-ad12-4a13-8ec7-897e9cb884b9 admin admin] PATH_INFO: `/v3/users/efadf6027540453a975d3dadb3f676c5` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:34.749065 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220443be-ad12-4a13-8ec7-897e9cb884b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efadf6027540453a975d3dadb3f676c5)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:34.751307 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220443be-ad12-4a13-8ec7-897e9cb884b9 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:34.863317 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-220443be-ad12-4a13-8ec7-897e9cb884b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efadf6027540453a975d3dadb3f676c5'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:34.864613 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-220443be-ad12-4a13-8ec7-897e9cb884b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efadf6027540453a975d3dadb3f676c5'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:34.887577 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-220443be-ad12-4a13-8ec7-897e9cb884b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efadf6027540453a975d3dadb3f676c5'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:34.889146 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-220443be-ad12-4a13-8ec7-897e9cb884b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efadf6027540453a975d3dadb3f676c5'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:34.899145 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-220443be-ad12-4a13-8ec7-897e9cb884b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efadf6027540453a975d3dadb3f676c5'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:34.923026 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1412/2820] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:46:34 2022] DELETE /identity/v3/users/efadf6027540453a975d3dadb3f676c5 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:35.041653 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406d842-4127-4eb9-8b04-8d5ab4b55ad3 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:35.042390 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406d842-4127-4eb9-8b04-8d5ab4b55ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:35.042390 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406d842-4127-4eb9-8b04-8d5ab4b55ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:35.078431 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-f406d842-4127-4eb9-8b04-8d5ab4b55ad3 None None] MFA Rules not processed for user `f3eb282e55c941ef9fefb57e14a708f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:35.080997 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f406d842-4127-4eb9-8b04-8d5ab4b55ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:35.153377 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220443be-ad12-4a13-8ec7-897e9cb884b9 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:35.156212 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-220443be-ad12-4a13-8ec7-897e9cb884b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:35.173162 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:35.222459 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1409/2821] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:46:35.225433 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f406d842-4127-4eb9-8b04-8d5ab4b55ad3 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:35.229370 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f406d842-4127-4eb9-8b04-8d5ab4b55ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:35.277413 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220443be-ad12-4a13-8ec7-897e9cb884b9 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:35.282352 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e804f-b495-4108-9fa0-24f1e40f4b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:35.283261 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e804f-b495-4108-9fa0-24f1e40f4b8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:35.283549 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e804f-b495-4108-9fa0-24f1e40f4b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:35.283828 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e804f-b495-4108-9fa0-24f1e40f4b8e admin admin] PATH_INFO: `/v3/projects/aafeedc33991462da96f4c238fe14256` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:35.285073 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7e804f-b495-4108-9fa0-24f1e40f4b8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=aafeedc33991462da96f4c238fe14256)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:35.287055 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7e804f-b495-4108-9fa0-24f1e40f4b8e admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:35.315377 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f406d842-4127-4eb9-8b04-8d5ab4b55ad3 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:35.320413 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d8e735-97e9-4f57-9192-f77834f4ec86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:35.321098 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d8e735-97e9-4f57-9192-f77834f4ec86 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:35.321389 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d8e735-97e9-4f57-9192-f77834f4ec86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:35.321660 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d8e735-97e9-4f57-9192-f77834f4ec86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:35.322423 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d8e735-97e9-4f57-9192-f77834f4ec86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:35.324735 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d8e735-97e9-4f57-9192-f77834f4ec86 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:35.366559 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-4f7e804f-b495-4108-9fa0-24f1e40f4b8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aafeedc33991462da96f4c238fe14256', 'request_id': 'req-4f7e804f-b495-4108-9fa0-24f1e40f4b8e'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:35.370861 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1410/2822] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:46:35 2022] POST /identity/v3/projects => generated 377 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:46:35.374613 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d8e735-97e9-4f57-9192-f77834f4ec86 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:35.381774 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-4f7e804f-b495-4108-9fa0-24f1e40f4b8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aafeedc33991462da96f4c238fe14256', 'request_id': 'req-4f7e804f-b495-4108-9fa0-24f1e40f4b8e'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:35.389611 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-4f7e804f-b495-4108-9fa0-24f1e40f4b8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aafeedc33991462da96f4c238fe14256', 'request_id': 'req-4f7e804f-b495-4108-9fa0-24f1e40f4b8e'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:35.412586 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-4f7e804f-b495-4108-9fa0-24f1e40f4b8e admin admin] The token cache is being invalidate because project aafeedc33991462da96f4c238fe14256 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:46:35.412968 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-4f7e804f-b495-4108-9fa0-24f1e40f4b8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:35.414022 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-4f7e804f-b495-4108-9fa0-24f1e40f4b8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:35.429516 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1413/2823] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:46:35 2022] DELETE /identity/v3/projects/aafeedc33991462da96f4c238fe14256 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:35.431107 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e0a5d-95ea-4999-9907-9a0a13acbec5 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:35.431408 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e0a5d-95ea-4999-9907-9a0a13acbec5 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:35.431692 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e0a5d-95ea-4999-9907-9a0a13acbec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:35.452573 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-4a0e0a5d-95ea-4999-9907-9a0a13acbec5 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:35.453794 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-4a0e0a5d-95ea-4999-9907-9a0a13acbec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:35.465244 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1414/2824] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:46:35.465385 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d8e735-97e9-4f57-9192-f77834f4ec86 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:35.469977 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e0ae0-b53e-4389-9090-aff271406361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:35.470854 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e0ae0-b53e-4389-9090-aff271406361 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:35.471206 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e0ae0-b53e-4389-9090-aff271406361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:35.471489 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e0ae0-b53e-4389-9090-aff271406361 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:35.472197 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8e0ae0-b53e-4389-9090-aff271406361 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:35.473513 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe6134-d87f-4d8f-a4d5-a77dddfadc2c None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:35.473875 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe6134-d87f-4d8f-a4d5-a77dddfadc2c None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:35.474196 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8e0ae0-b53e-4389-9090-aff271406361 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:35.474323 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe6134-d87f-4d8f-a4d5-a77dddfadc2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:35.484374 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:35.497397 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-d9fe6134-d87f-4d8f-a4d5-a77dddfadc2c None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:35.498674 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-d9fe6134-d87f-4d8f-a4d5-a77dddfadc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:35.503075 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1411/2825] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:46:35 2022] POST /identity/v3/users => generated 295 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:46:35.506549 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e0ae0-b53e-4389-9090-aff271406361 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:35.509477 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-fb8e0ae0-b53e-4389-9090-aff271406361 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:35.510191 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1415/2826] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:46:35.515100 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe6134-d87f-4d8f-a4d5-a77dddfadc2c None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:35.524997 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e0ae0-b53e-4389-9090-aff271406361 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:35.526523 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe6134-d87f-4d8f-a4d5-a77dddfadc2c None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:35.530256 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3709e04e-fb14-4039-b18e-304ee0b7893e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:35.530990 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3709e04e-fb14-4039-b18e-304ee0b7893e admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:35.531286 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3709e04e-fb14-4039-b18e-304ee0b7893e admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:35.531556 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3709e04e-fb14-4039-b18e-304ee0b7893e admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:35.531689 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c86b9-4ad7-4d89-accc-2b3e2b01ac55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:35.532423 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c86b9-4ad7-4d89-accc-2b3e2b01ac55 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:35.532605 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3709e04e-fb14-4039-b18e-304ee0b7893e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:35.532829 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c86b9-4ad7-4d89-accc-2b3e2b01ac55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:35.533184 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c86b9-4ad7-4d89-accc-2b3e2b01ac55 admin admin] PATH_INFO: `/v3/domains` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:35.534010 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8c86b9-4ad7-4d89-accc-2b3e2b01ac55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80223) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 19:46:35.534559 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8c86b9-4ad7-4d89-accc-2b3e2b01ac55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:35.535531 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3709e04e-fb14-4039-b18e-304ee0b7893e admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:35.536486 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8c86b9-4ad7-4d89-accc-2b3e2b01ac55 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:35.547525 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1412/2827] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:46:35.554871 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3709e04e-fb14-4039-b18e-304ee0b7893e admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:35.563073 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1416/2828] 10.222.0.15 () {58 vars in 1152 bytes} [Wed Jun 15 19:46:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 19:46:35.564382 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c86b9-4ad7-4d89-accc-2b3e2b01ac55 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:35.568815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3709e04e-fb14-4039-b18e-304ee0b7893e admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:35.576635 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c86b9-4ad7-4d89-accc-2b3e2b01ac55 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:35.578076 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f9a0f-ff88-4aef-93da-69b21188949a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:35.579852 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f9a0f-ff88-4aef-93da-69b21188949a admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:35.580368 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f9a0f-ff88-4aef-93da-69b21188949a admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:35.580601 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f9a0f-ff88-4aef-93da-69b21188949a admin admin] PATH_INFO: `/v3/projects/f36196d5f3d049eb8cd537ff68322c00/users/3c085f1659164832949fe79b67b88625/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:35.582286 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3f2583-5081-4da0-8cb0-aa4d1909a470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:35.583029 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f2583-5081-4da0-8cb0-aa4d1909a470 admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:35.583328 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f2583-5081-4da0-8cb0-aa4d1909a470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:35.583676 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f2583-5081-4da0-8cb0-aa4d1909a470 admin admin] PATH_INFO: `/v3/projects` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:35.584478 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3f2583-5081-4da0-8cb0-aa4d1909a470 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:35.586513 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3f2583-5081-4da0-8cb0-aa4d1909a470 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:35.605316 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f9a0f-ff88-4aef-93da-69b21188949a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f36196d5f3d049eb8cd537ff68322c00, user_id=3c085f1659164832949fe79b67b88625, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:35.608049 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f9a0f-ff88-4aef-93da-69b21188949a admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:35.626473 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1413/2829] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:46:35 2022] PUT /identity/v3/projects/f36196d5f3d049eb8cd537ff68322c00/users/3c085f1659164832949fe79b67b88625/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:35.626969 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1417/2830] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:46:35 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:46:35.627703 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f9a0f-ff88-4aef-93da-69b21188949a admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:35.631577 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3f2583-5081-4da0-8cb0-aa4d1909a470 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:35.693950 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:35.714318 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3f2583-5081-4da0-8cb0-aa4d1909a470 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:35.719938 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f9a0f-ff88-4aef-93da-69b21188949a admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:35.720399 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c4dad-7428-4dde-9af0-08d06f7ffafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:35.721092 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c4dad-7428-4dde-9af0-08d06f7ffafb admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:35.721375 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c4dad-7428-4dde-9af0-08d06f7ffafb admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:35.721651 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c4dad-7428-4dde-9af0-08d06f7ffafb admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:35.722390 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8c4dad-7428-4dde-9af0-08d06f7ffafb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:35.724507 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8c4dad-7428-4dde-9af0-08d06f7ffafb admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:35.728355 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12d770-83cd-4e27-961a-85097f7e3de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:35.729568 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12d770-83cd-4e27-961a-85097f7e3de4 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:35.730250 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12d770-83cd-4e27-961a-85097f7e3de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:35.730912 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12d770-83cd-4e27-961a-85097f7e3de4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:35.732494 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc12d770-83cd-4e27-961a-85097f7e3de4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:35.736264 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc12d770-83cd-4e27-961a-85097f7e3de4 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:35.750663 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1418/2831] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:46:35 2022] POST /identity/v3/users => generated 293 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:46:35.753168 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1414/2832] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:46:35.756225 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c4dad-7428-4dde-9af0-08d06f7ffafb admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:35.760283 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12d770-83cd-4e27-961a-85097f7e3de4 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:35.768114 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c4dad-7428-4dde-9af0-08d06f7ffafb admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:35.775734 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0913f6-144f-4ca4-8046-0bc5ee1ee2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:35.776762 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0913f6-144f-4ca4-8046-0bc5ee1ee2a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:35.777231 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0913f6-144f-4ca4-8046-0bc5ee1ee2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:35.777700 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0913f6-144f-4ca4-8046-0bc5ee1ee2a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:35.779044 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0913f6-144f-4ca4-8046-0bc5ee1ee2a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:35.779827 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12d770-83cd-4e27-961a-85097f7e3de4 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:35.781898 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0913f6-144f-4ca4-8046-0bc5ee1ee2a0 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:35.787766 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f924d460-68d5-4f06-a0e5-97e0bfa181e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:35.789211 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924d460-68d5-4f06-a0e5-97e0bfa181e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:35.789711 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924d460-68d5-4f06-a0e5-97e0bfa181e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:35.790205 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924d460-68d5-4f06-a0e5-97e0bfa181e8 admin admin] PATH_INFO: `/v3/projects/f36196d5f3d049eb8cd537ff68322c00/users/3c085f1659164832949fe79b67b88625/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:35.794064 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f924d460-68d5-4f06-a0e5-97e0bfa181e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f36196d5f3d049eb8cd537ff68322c00, user_id=3c085f1659164832949fe79b67b88625, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:35.798897 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f924d460-68d5-4f06-a0e5-97e0bfa181e8 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:35.800463 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1419/2833] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 19:46:35.807312 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0913f6-144f-4ca4-8046-0bc5ee1ee2a0 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:35.815277 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1415/2834] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:46:35 2022] PUT /identity/v3/projects/f36196d5f3d049eb8cd537ff68322c00/users/3c085f1659164832949fe79b67b88625/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:35.822541 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0913f6-144f-4ca4-8046-0bc5ee1ee2a0 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:35.901582 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bca9fdf-d008-4e91-b2ad-a9a7164fa414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:35.902282 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca9fdf-d008-4e91-b2ad-a9a7164fa414 admin admin] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:35.902567 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca9fdf-d008-4e91-b2ad-a9a7164fa414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:35.902857 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca9fdf-d008-4e91-b2ad-a9a7164fa414 admin admin] PATH_INFO: `/v3/roles` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:35.904885 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bca9fdf-d008-4e91-b2ad-a9a7164fa414 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:35.906629 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bca9fdf-d008-4e91-b2ad-a9a7164fa414 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:35.936253 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1420/2835] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:46:35.944867 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f924d460-68d5-4f06-a0e5-97e0bfa181e8 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:35.967507 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f924d460-68d5-4f06-a0e5-97e0bfa181e8 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:35.984548 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8630d4-a83f-4e79-9627-a38d90fc27f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:35.987200 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8630d4-a83f-4e79-9627-a38d90fc27f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:35.987574 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8630d4-a83f-4e79-9627-a38d90fc27f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:35.987785 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8630d4-a83f-4e79-9627-a38d90fc27f2 admin admin] PATH_INFO: `/v3/projects/5eeb6d09ac2a459183dac0e66a26b4bc/users/841ccfe45fc64b15b97426ee692deb5b/roles/e53a78912e764f8387e0cf637bb86bee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:36.019141 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8630d4-a83f-4e79-9627-a38d90fc27f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eeb6d09ac2a459183dac0e66a26b4bc, user_id=841ccfe45fc64b15b97426ee692deb5b, role_id=e53a78912e764f8387e0cf637bb86bee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:36.022417 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8630d4-a83f-4e79-9627-a38d90fc27f2 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:36.038619 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1416/2836] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:46:35 2022] PUT /identity/v3/projects/5eeb6d09ac2a459183dac0e66a26b4bc/users/841ccfe45fc64b15b97426ee692deb5b/roles/e53a78912e764f8387e0cf637bb86bee => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:36.047886 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729b44f-8f40-477c-a17f-717324fec1b1 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:36.047886 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729b44f-8f40-477c-a17f-717324fec1b1 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:36.047886 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729b44f-8f40-477c-a17f-717324fec1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:36.066800 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-1729b44f-8f40-477c-a17f-717324fec1b1 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:36.068171 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-1729b44f-8f40-477c-a17f-717324fec1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:36.154339 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1421/2837] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:46:36.164103 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8630d4-a83f-4e79-9627-a38d90fc27f2 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:36.166585 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-db8630d4-a83f-4e79-9627-a38d90fc27f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:36.237799 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8630d4-a83f-4e79-9627-a38d90fc27f2 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:36.244659 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4961a34-8e3a-4cdc-98b5-b379355a8850 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:36.245359 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4961a34-8e3a-4cdc-98b5-b379355a8850 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:36.245688 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4961a34-8e3a-4cdc-98b5-b379355a8850 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:36.246010 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4961a34-8e3a-4cdc-98b5-b379355a8850 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:36.257683 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4961a34-8e3a-4cdc-98b5-b379355a8850 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:36.260061 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4961a34-8e3a-4cdc-98b5-b379355a8850 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:36.273190 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1417/2838] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:46:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:46:42.300785 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729b44f-8f40-477c-a17f-717324fec1b1 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:42.303644 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-1729b44f-8f40-477c-a17f-717324fec1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:42.323732 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729b44f-8f40-477c-a17f-717324fec1b1 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:42.336633 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8797a4-e45b-4af3-86a7-1506c9496bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:42.337816 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8797a4-e45b-4af3-86a7-1506c9496bc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:42.338514 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8797a4-e45b-4af3-86a7-1506c9496bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:42.338999 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8797a4-e45b-4af3-86a7-1506c9496bc1 admin admin] PATH_INFO: `/v3/users/d3f449902fcf47b4b61158cda7c42eaf` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:42.341462 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8797a4-e45b-4af3-86a7-1506c9496bc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3f449902fcf47b4b61158cda7c42eaf)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:42.344108 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8797a4-e45b-4af3-86a7-1506c9496bc1 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:42.422375 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-fd8797a4-e45b-4af3-86a7-1506c9496bc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3f449902fcf47b4b61158cda7c42eaf'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:42.423035 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-fd8797a4-e45b-4af3-86a7-1506c9496bc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3f449902fcf47b4b61158cda7c42eaf'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:42.438048 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-fd8797a4-e45b-4af3-86a7-1506c9496bc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3f449902fcf47b4b61158cda7c42eaf'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:42.440543 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-fd8797a4-e45b-4af3-86a7-1506c9496bc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3f449902fcf47b4b61158cda7c42eaf'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:42.467076 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:42.477751 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-fd8797a4-e45b-4af3-86a7-1506c9496bc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3f449902fcf47b4b61158cda7c42eaf'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:42.522470 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1422/2839] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:46:42 2022] DELETE /identity/v3/users/d3f449902fcf47b4b61158cda7c42eaf => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:42.524850 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea9e0a-b158-4361-aaa3-fbaf99cf9645 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:42.525332 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea9e0a-b158-4361-aaa3-fbaf99cf9645 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:42.525447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea9e0a-b158-4361-aaa3-fbaf99cf9645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:42.553884 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-59ea9e0a-b158-4361-aaa3-fbaf99cf9645 None None] MFA Rules not processed for user `d018cc34dbeb4d56a3052de5c20b961c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:42.556085 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-59ea9e0a-b158-4361-aaa3-fbaf99cf9645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:42.572548 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:42.651304 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1418/2840] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:46:42.659424 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8797a4-e45b-4af3-86a7-1506c9496bc1 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:42.661082 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-fd8797a4-e45b-4af3-86a7-1506c9496bc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:42.762333 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8797a4-e45b-4af3-86a7-1506c9496bc1 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:42.771996 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5851079-de52-4c81-b8ff-b8f984b341e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:42.772881 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5851079-de52-4c81-b8ff-b8f984b341e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:42.773343 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5851079-de52-4c81-b8ff-b8f984b341e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:42.773811 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5851079-de52-4c81-b8ff-b8f984b341e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:42.786394 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5851079-de52-4c81-b8ff-b8f984b341e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:42.788349 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5851079-de52-4c81-b8ff-b8f984b341e3 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:42.801006 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1423/2841] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:46:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:46:43.274567 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea9e0a-b158-4361-aaa3-fbaf99cf9645 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:43.277374 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-59ea9e0a-b158-4361-aaa3-fbaf99cf9645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:43.290655 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ced95-db18-44f5-af3f-f42bdc33a6b0 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:43.290949 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ced95-db18-44f5-af3f-f42bdc33a6b0 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:43.291236 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ced95-db18-44f5-af3f-f42bdc33a6b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:43.314903 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea9e0a-b158-4361-aaa3-fbaf99cf9645 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:43.331366 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14de830-21fb-48ae-ba38-e13a7fbf9183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:43.332289 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14de830-21fb-48ae-ba38-e13a7fbf9183 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:43.332921 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14de830-21fb-48ae-ba38-e13a7fbf9183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:43.332921 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14de830-21fb-48ae-ba38-e13a7fbf9183 admin admin] PATH_INFO: `/v3/projects/46f596955a244cb69437b9ccc5668239` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:43.334253 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14de830-21fb-48ae-ba38-e13a7fbf9183 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46f596955a244cb69437b9ccc5668239)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:43.336365 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14de830-21fb-48ae-ba38-e13a7fbf9183 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:43.364031 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-a78ced95-db18-44f5-af3f-f42bdc33a6b0 None None] MFA Rules not processed for user `3c085f1659164832949fe79b67b88625`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:43.366459 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-a78ced95-db18-44f5-af3f-f42bdc33a6b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:43.454183 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-d14de830-21fb-48ae-ba38-e13a7fbf9183 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46f596955a244cb69437b9ccc5668239', 'request_id': 'req-d14de830-21fb-48ae-ba38-e13a7fbf9183'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:43.473702 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-d14de830-21fb-48ae-ba38-e13a7fbf9183 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46f596955a244cb69437b9ccc5668239', 'request_id': 'req-d14de830-21fb-48ae-ba38-e13a7fbf9183'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:43.484755 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-d14de830-21fb-48ae-ba38-e13a7fbf9183 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46f596955a244cb69437b9ccc5668239', 'request_id': 'req-d14de830-21fb-48ae-ba38-e13a7fbf9183'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:43.499477 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-d14de830-21fb-48ae-ba38-e13a7fbf9183 admin admin] The token cache is being invalidate because project 46f596955a244cb69437b9ccc5668239 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:46:43.500086 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-d14de830-21fb-48ae-ba38-e13a7fbf9183 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:43.501091 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-d14de830-21fb-48ae-ba38-e13a7fbf9183 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:46:43.503391 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1419/2842] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:46:43 2022] DELETE /identity/v3/projects/46f596955a244cb69437b9ccc5668239 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:43.538213 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1424/2843] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:43 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 19:46:43.549926 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14de830-21fb-48ae-ba38-e13a7fbf9183 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:43.556792 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d14de830-21fb-48ae-ba38-e13a7fbf9183 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:43.782871 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14de830-21fb-48ae-ba38-e13a7fbf9183 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:43.789975 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2fee0b-730f-4aca-b374-9eb83f6392fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:43.791148 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2fee0b-730f-4aca-b374-9eb83f6392fb service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:43.791659 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2fee0b-730f-4aca-b374-9eb83f6392fb service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:43.792330 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2fee0b-730f-4aca-b374-9eb83f6392fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:43.794930 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-fa2fee0b-730f-4aca-b374-9eb83f6392fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:43.884648 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:43.929398 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2fee0b-730f-4aca-b374-9eb83f6392fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:43.932127 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2fee0b-730f-4aca-b374-9eb83f6392fb service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:43.944948 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1420/2844] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:46:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:46:44.328754 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ae3f0-d8ef-431a-9db5-f02c63bd0f34 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:44.329937 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ae3f0-d8ef-431a-9db5-f02c63bd0f34 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:44.330430 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ae3f0-d8ef-431a-9db5-f02c63bd0f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:44.373880 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-770ae3f0-d8ef-431a-9db5-f02c63bd0f34 None None] MFA Rules not processed for user `841ccfe45fc64b15b97426ee692deb5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:44.375943 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-770ae3f0-d8ef-431a-9db5-f02c63bd0f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:44.477529 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:44.497799 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1425/2845] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:46:44.502224 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2fee0b-730f-4aca-b374-9eb83f6392fb service nova] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:44.504045 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-fa2fee0b-730f-4aca-b374-9eb83f6392fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:44.618361 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2fee0b-730f-4aca-b374-9eb83f6392fb service nova] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:44.624490 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981430a-58c4-4e6e-86ff-31a9b7dd05de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:44.625331 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981430a-58c4-4e6e-86ff-31a9b7dd05de admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:44.626319 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981430a-58c4-4e6e-86ff-31a9b7dd05de admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:44.626598 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981430a-58c4-4e6e-86ff-31a9b7dd05de admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:44.627594 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981430a-58c4-4e6e-86ff-31a9b7dd05de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:44.629669 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981430a-58c4-4e6e-86ff-31a9b7dd05de admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:44.664982 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1421/2846] 10.222.0.15 () {60 vars in 1152 bytes} [Wed Jun 15 19:46:44 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 19:46:44.669590 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ae3f0-d8ef-431a-9db5-f02c63bd0f34 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:44.692453 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ae3f0-d8ef-431a-9db5-f02c63bd0f34 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:44.783811 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392938e-9eab-4f31-9fe5-e64762dd2d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:44.784184 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392938e-9eab-4f31-9fe5-e64762dd2d9c admin admin] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:44.784477 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392938e-9eab-4f31-9fe5-e64762dd2d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:44.784861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392938e-9eab-4f31-9fe5-e64762dd2d9c admin admin] PATH_INFO: `/v3/users` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:44.785584 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8392938e-9eab-4f31-9fe5-e64762dd2d9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:44.787860 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8392938e-9eab-4f31-9fe5-e64762dd2d9c admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:44.853118 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1426/2847] 10.222.0.15 () {60 vars in 1143 bytes} [Wed Jun 15 19:46:44 2022] POST /identity/v3/users => generated 299 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 19:46:44.855604 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981430a-58c4-4e6e-86ff-31a9b7dd05de admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:44.867116 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981430a-58c4-4e6e-86ff-31a9b7dd05de admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:44.872824 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398cdac-2bb2-4af0-8781-fd74b2256d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:44.873511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398cdac-2bb2-4af0-8781-fd74b2256d30 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:44.873796 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398cdac-2bb2-4af0-8781-fd74b2256d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:44.874073 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398cdac-2bb2-4af0-8781-fd74b2256d30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:44.874908 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4398cdac-2bb2-4af0-8781-fd74b2256d30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:44.876573 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4398cdac-2bb2-4af0-8781-fd74b2256d30 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:44.889089 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1422/2848] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:46:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:46:44.893595 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392938e-9eab-4f31-9fe5-e64762dd2d9c admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:44.910790 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392938e-9eab-4f31-9fe5-e64762dd2d9c admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:44.916704 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe4f717-cd4b-4ff6-b211-59d1ebeef94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:44.918007 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4f717-cd4b-4ff6-b211-59d1ebeef94b admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:44.918495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4f717-cd4b-4ff6-b211-59d1ebeef94b admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:44.919037 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4f717-cd4b-4ff6-b211-59d1ebeef94b admin admin] PATH_INFO: `/v3/projects/5ad09f273d794c659d2c90c019b4aeeb/users/7e83ce09589645138e7ef934cbd98d5d/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:44.953088 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe4f717-cd4b-4ff6-b211-59d1ebeef94b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ad09f273d794c659d2c90c019b4aeeb, user_id=7e83ce09589645138e7ef934cbd98d5d, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:44.955808 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe4f717-cd4b-4ff6-b211-59d1ebeef94b admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:44.974517 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1427/2849] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:46:44 2022] PUT /identity/v3/projects/5ad09f273d794c659d2c90c019b4aeeb/users/7e83ce09589645138e7ef934cbd98d5d/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:46:44.975744 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398cdac-2bb2-4af0-8781-fd74b2256d30 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:44.986603 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398cdac-2bb2-4af0-8781-fd74b2256d30 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:45.075955 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099f804c-7caa-46e2-8c53-688273f33bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:45.077037 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f804c-7caa-46e2-8c53-688273f33bf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:45.077492 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f804c-7caa-46e2-8c53-688273f33bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:45.077918 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f804c-7caa-46e2-8c53-688273f33bf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:45.078883 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099f804c-7caa-46e2-8c53-688273f33bf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:45.081042 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099f804c-7caa-46e2-8c53-688273f33bf8 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:45.099283 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1423/2850] 10.222.0.15 () {58 vars in 1121 bytes} [Wed Jun 15 19:46:44 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 19:46:45.106013 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe4f717-cd4b-4ff6-b211-59d1ebeef94b admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:45.118977 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe4f717-cd4b-4ff6-b211-59d1ebeef94b admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:45.125877 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d7420-9544-43f4-b1a0-d3f833b0a2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:45.126858 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d7420-9544-43f4-b1a0-d3f833b0a2fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:45.127139 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d7420-9544-43f4-b1a0-d3f833b0a2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:45.127413 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d7420-9544-43f4-b1a0-d3f833b0a2fd admin admin] PATH_INFO: `/v3/projects/5ad09f273d794c659d2c90c019b4aeeb/users/7e83ce09589645138e7ef934cbd98d5d/roles/4f736313c26d4f8db48bbb2f346e50e4` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:45.129561 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409d7420-9544-43f4-b1a0-d3f833b0a2fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ad09f273d794c659d2c90c019b4aeeb, user_id=7e83ce09589645138e7ef934cbd98d5d, role_id=4f736313c26d4f8db48bbb2f346e50e4)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:45.132494 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409d7420-9544-43f4-b1a0-d3f833b0a2fd admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:45.145283 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1428/2851] 10.222.0.15 () {60 vars in 1482 bytes} [Wed Jun 15 19:46:45 2022] PUT /identity/v3/projects/5ad09f273d794c659d2c90c019b4aeeb/users/7e83ce09589645138e7ef934cbd98d5d/roles/4f736313c26d4f8db48bbb2f346e50e4 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 19:46:50.411040 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1016761-45ba-4e00-98f9-0f8f3fe7cfff None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:50.411497 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1016761-45ba-4e00-98f9-0f8f3fe7cfff None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:50.411673 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1016761-45ba-4e00-98f9-0f8f3fe7cfff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:50.453759 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-a1016761-45ba-4e00-98f9-0f8f3fe7cfff None None] MFA Rules not processed for user `7e83ce09589645138e7ef934cbd98d5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:50.455068 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a1016761-45ba-4e00-98f9-0f8f3fe7cfff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:50.534815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:50.597798 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1424/2852] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:46:50 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:46:50.609106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d7420-9544-43f4-b1a0-d3f833b0a2fd admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:50.610982 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-409d7420-9544-43f4-b1a0-d3f833b0a2fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:50.780618 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d7420-9544-43f4-b1a0-d3f833b0a2fd admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:50.787325 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d549d-9747-4bcb-8011-686317f72d5c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfca4d302aa43249609513f35d55d7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:50.790335 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d549d-9747-4bcb-8011-686317f72d5c service glance] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:50.790335 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d549d-9747-4bcb-8011-686317f72d5c service glance] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:50.790335 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d549d-9747-4bcb-8011-686317f72d5c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:50.936064 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2d549d-9747-4bcb-8011-686317f72d5c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:50.937898 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2d549d-9747-4bcb-8011-686317f72d5c service glance] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:50.948577 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:46:50.955026 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1429/2853] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:46:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:46:51.126963 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6f7af-271d-4ba1-80a5-ac9756a7f9a2 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:51.127253 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6f7af-271d-4ba1-80a5-ac9756a7f9a2 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:51.127567 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6f7af-271d-4ba1-80a5-ac9756a7f9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:51.151813 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-1ed6f7af-271d-4ba1-80a5-ac9756a7f9a2 None None] MFA Rules not processed for user `67aa048d409c46e9a729aec9d1b2286e`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:46:51.153393 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1ed6f7af-271d-4ba1-80a5-ac9756a7f9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:51.232473 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1425/2854] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Jun 15 19:46:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:46:51.246745 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d549d-9747-4bcb-8011-686317f72d5c service glance] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:46:51.248533 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-7a2d549d-9747-4bcb-8011-686317f72d5c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:46:51.348073 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d549d-9747-4bcb-8011-686317f72d5c service glance] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:46:51.354127 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2a258-a492-4118-84fc-060265569c0f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176d5775a70f4524a9b7fe1ce9d426ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:46:51.354889 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2a258-a492-4118-84fc-060265569c0f service swift] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:46:51.355181 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2a258-a492-4118-84fc-060265569c0f service swift] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:46:51.355466 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2a258-a492-4118-84fc-060265569c0f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:46:51.369282 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd2a258-a492-4118-84fc-060265569c0f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:46:51.372773 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd2a258-a492-4118-84fc-060265569c0f service swift] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:46:51.390042 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1430/2855] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Jun 15 19:46:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 19:47:00.771441 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86961b3c-8487-48dd-8866-823bf62bc46c None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:47:00.771986 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86961b3c-8487-48dd-8866-823bf62bc46c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:47:00.771986 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86961b3c-8487-48dd-8866-823bf62bc46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:47:00.790974 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-86961b3c-8487-48dd-8866-823bf62bc46c None None] MFA Rules not processed for user `841ccfe45fc64b15b97426ee692deb5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:47:00.792180 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-86961b3c-8487-48dd-8866-823bf62bc46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:47:00.799086 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1426/2856] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:47:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:47:01.099879 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add5f42-6020-46b9-b3c7-ed2262e80622 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:47:01.100423 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add5f42-6020-46b9-b3c7-ed2262e80622 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:47:01.100423 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add5f42-6020-46b9-b3c7-ed2262e80622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:47:01.119282 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-3add5f42-6020-46b9-b3c7-ed2262e80622 None None] MFA Rules not processed for user `f3eb282e55c941ef9fefb57e14a708f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:47:01.120527 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-3add5f42-6020-46b9-b3c7-ed2262e80622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:47:01.182167 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1431/2857] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:47:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:47:01.574296 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86961b3c-8487-48dd-8866-823bf62bc46c None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:47:01.575906 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-86961b3c-8487-48dd-8866-823bf62bc46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:47:01.644534 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86961b3c-8487-48dd-8866-823bf62bc46c None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:47:01.648449 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12c84d-e2d6-4bf7-ba4f-d17f583aade4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:47:01.649259 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12c84d-e2d6-4bf7-ba4f-d17f583aade4 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:47:01.649625 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12c84d-e2d6-4bf7-ba4f-d17f583aade4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:47:01.650010 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12c84d-e2d6-4bf7-ba4f-d17f583aade4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:47:01.653349 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-dc12c84d-e2d6-4bf7-ba4f-d17f583aade4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:47:01.665361 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc12c84d-e2d6-4bf7-ba4f-d17f583aade4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:47:01.666889 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc12c84d-e2d6-4bf7-ba4f-d17f583aade4 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:47:01.676717 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1427/2858] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:47:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 19:47:02.665713 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769214e-5cab-41d4-b28a-66f6a4d2b986 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:47:02.666284 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769214e-5cab-41d4-b28a-66f6a4d2b986 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:47:02.666588 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769214e-5cab-41d4-b28a-66f6a4d2b986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:47:02.687343 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-2769214e-5cab-41d4-b28a-66f6a4d2b986 None None] MFA Rules not processed for user `67aa048d409c46e9a729aec9d1b2286e`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:47:02.688806 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-2769214e-5cab-41d4-b28a-66f6a4d2b986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:47:02.696455 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1432/2859] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Jun 15 19:47:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:47:02.713030 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12c84d-e2d6-4bf7-ba4f-d17f583aade4 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:47:02.727222 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12c84d-e2d6-4bf7-ba4f-d17f583aade4 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:47:02.732385 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44567e2a-545c-4679-8c56-869e35836386 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176d5775a70f4524a9b7fe1ce9d426ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:47:02.733115 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44567e2a-545c-4679-8c56-869e35836386 service swift] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:47:02.733401 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44567e2a-545c-4679-8c56-869e35836386 service swift] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:47:02.733678 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44567e2a-545c-4679-8c56-869e35836386 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:47:02.747340 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44567e2a-545c-4679-8c56-869e35836386 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:47:02.747340 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44567e2a-545c-4679-8c56-869e35836386 service swift] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:47:02.755986 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1428/2860] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Jun 15 19:47:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 19:47:12.681055 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=60 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:47:12.752674 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=72 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:47:24.979071 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570efc0a-9b1d-43d0-ab36-036be8412efd None None] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:47:24.979895 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570efc0a-9b1d-43d0-ab36-036be8412efd None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:47:24.979895 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570efc0a-9b1d-43d0-ab36-036be8412efd None None] PATH_INFO: `/` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:47:24.982072 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1433/2861] 10.222.0.15 () {54 vars in 921 bytes} [Wed Jun 15 19:47:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:47:24.988046 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e4bae-107e-4cb1-95f2-e60a05400276 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:47:24.988590 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e4bae-107e-4cb1-95f2-e60a05400276 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:47:24.988590 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e4bae-107e-4cb1-95f2-e60a05400276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:47:25.009276 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-357e4bae-107e-4cb1-95f2-e60a05400276 None None] MFA Rules not processed for user `5c63e0b6ceed4f7cb2e6f9269eb85795`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:47:25.010929 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-357e4bae-107e-4cb1-95f2-e60a05400276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:47:25.018165 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1429/2862] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Jun 15 19:47:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:47:25.032088 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570efc0a-9b1d-43d0-ab36-036be8412efd None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:47:25.033969 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-570efc0a-9b1d-43d0-ab36-036be8412efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:47:25.048599 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570efc0a-9b1d-43d0-ab36-036be8412efd None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:47:25.052193 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d14b4d0-85b3-4309-91ce-9c2a40a25a42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:47:25.052891 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14b4d0-85b3-4309-91ce-9c2a40a25a42 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:47:25.053178 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14b4d0-85b3-4309-91ce-9c2a40a25a42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:47:25.053454 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14b4d0-85b3-4309-91ce-9c2a40a25a42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:47:25.061725 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d14b4d0-85b3-4309-91ce-9c2a40a25a42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:47:25.063201 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d14b4d0-85b3-4309-91ce-9c2a40a25a42 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:47:25.072713 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1434/2863] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:47:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:47:25.158872 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8739a-69dd-4a49-bc72-7282676bb0a3 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:47:25.159162 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8739a-69dd-4a49-bc72-7282676bb0a3 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:47:25.159439 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8739a-69dd-4a49-bc72-7282676bb0a3 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:47:25.160818 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1430/2864] 10.222.0.15 () {54 vars in 921 bytes} [Wed Jun 15 19:47:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 19:47:25.166363 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c00bad-854d-48dd-b77a-72f86faca9af None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:47:25.166652 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c00bad-854d-48dd-b77a-72f86faca9af None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:47:25.166927 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c00bad-854d-48dd-b77a-72f86faca9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:47:25.184870 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-b2c00bad-854d-48dd-b77a-72f86faca9af None None] MFA Rules not processed for user `3e74ac44e5814122a3226ac023f3688a`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:47:25.186071 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-b2c00bad-854d-48dd-b77a-72f86faca9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:47:25.255060 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1435/2865] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Jun 15 19:47:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:47:25.266029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8739a-69dd-4a49-bc72-7282676bb0a3 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:47:25.277754 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8739a-69dd-4a49-bc72-7282676bb0a3 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:47:25.281476 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ace793-e0a9-43a7-92f6-e0e0f41fc2e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:47:25.282153 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ace793-e0a9-43a7-92f6-e0e0f41fc2e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:47:25.282441 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ace793-e0a9-43a7-92f6-e0e0f41fc2e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:47:25.282719 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ace793-e0a9-43a7-92f6-e0e0f41fc2e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:47:25.293458 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ace793-e0a9-43a7-92f6-e0e0f41fc2e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:47:25.294921 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ace793-e0a9-43a7-92f6-e0e0f41fc2e8 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:47:25.306595 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1431/2866] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:47:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:47:28.456310 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c00bad-854d-48dd-b77a-72f86faca9af None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:47:28.466175 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c00bad-854d-48dd-b77a-72f86faca9af None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:47:28.469930 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef1242c-ded8-4219-b15f-575bc1a71c76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:47:28.470642 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef1242c-ded8-4219-b15f-575bc1a71c76 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:47:28.470964 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef1242c-ded8-4219-b15f-575bc1a71c76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:47:28.471240 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef1242c-ded8-4219-b15f-575bc1a71c76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:47:28.473099 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-eef1242c-ded8-4219-b15f-575bc1a71c76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:47:28.563980 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef1242c-ded8-4219-b15f-575bc1a71c76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:47:28.565468 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef1242c-ded8-4219-b15f-575bc1a71c76 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:47:28.577877 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1436/2867] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:47:28 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:47:30.177542 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ace793-e0a9-43a7-92f6-e0e0f41fc2e8 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:47:30.186804 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ace793-e0a9-43a7-92f6-e0e0f41fc2e8 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:47:30.192480 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af491d-a4f9-466b-9730-3c7d8fb4335f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:47:30.193923 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af491d-a4f9-466b-9730-3c7d8fb4335f service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:47:30.194209 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af491d-a4f9-466b-9730-3c7d8fb4335f service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:47:30.194491 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af491d-a4f9-466b-9730-3c7d8fb4335f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:47:30.205340 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96af491d-a4f9-466b-9730-3c7d8fb4335f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:47:30.207275 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96af491d-a4f9-466b-9730-3c7d8fb4335f service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:47:30.220467 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1432/2868] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:47:30 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:47:38.572071 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=67 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:47:40.214756 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=15 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:47:56.871833 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef1242c-ded8-4219-b15f-575bc1a71c76 service neutron] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:47:56.881106 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef1242c-ded8-4219-b15f-575bc1a71c76 service neutron] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:47:56.886019 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2bf006-41f2-4630-acf5-31b3b81767c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:47:56.886762 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2bf006-41f2-4630-acf5-31b3b81767c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:47:56.887133 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2bf006-41f2-4630-acf5-31b3b81767c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:47:56.887636 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2bf006-41f2-4630-acf5-31b3b81767c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:47:56.890490 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-0f2bf006-41f2-4630-acf5-31b3b81767c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:47:56.912232 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2bf006-41f2-4630-acf5-31b3b81767c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:47:56.914913 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2bf006-41f2-4630-acf5-31b3b81767c5 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:47:56.929861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1437/2869] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:47:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:47:58.120038 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af491d-a4f9-466b-9730-3c7d8fb4335f service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:47:58.128917 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af491d-a4f9-466b-9730-3c7d8fb4335f service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:47:58.132593 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250e47f-c086-4003-95c4-a13bd91a2712 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:47:58.133222 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250e47f-c086-4003-95c4-a13bd91a2712 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:47:58.133502 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250e47f-c086-4003-95c4-a13bd91a2712 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:47:58.133772 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250e47f-c086-4003-95c4-a13bd91a2712 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:47:58.135594 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-2250e47f-c086-4003-95c4-a13bd91a2712 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:47:58.149069 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2250e47f-c086-4003-95c4-a13bd91a2712 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:47:58.150681 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2250e47f-c086-4003-95c4-a13bd91a2712 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:47:58.161649 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1433/2870] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:47:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:48:06.922169 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=6 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:48:08.156724 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=6 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:48:42.161783 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2bf006-41f2-4630-acf5-31b3b81767c5 service neutron] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:48:42.163954 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-0f2bf006-41f2-4630-acf5-31b3b81767c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:48:42.193968 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2bf006-41f2-4630-acf5-31b3b81767c5 service neutron] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:48:42.217421 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfe865-fcba-464b-8b8c-dfc06a637389 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:48:42.218154 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfe865-fcba-464b-8b8c-dfc06a637389 service nova] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:48:42.218456 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfe865-fcba-464b-8b8c-dfc06a637389 service nova] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:48:42.218739 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfe865-fcba-464b-8b8c-dfc06a637389 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:48:42.220703 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-10bfe865-fcba-464b-8b8c-dfc06a637389 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:48:42.236157 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bfe865-fcba-464b-8b8c-dfc06a637389 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:48:42.237980 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bfe865-fcba-464b-8b8c-dfc06a637389 service nova] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:48:42.265682 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1438/2871] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:48:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:48:42.344000 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250e47f-c086-4003-95c4-a13bd91a2712 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:48:42.355924 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250e47f-c086-4003-95c4-a13bd91a2712 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:48:42.360014 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce31cc-4648-4566-9689-80ffb4b95a57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:48:42.360822 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce31cc-4648-4566-9689-80ffb4b95a57 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:48:42.361069 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce31cc-4648-4566-9689-80ffb4b95a57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:48:42.361401 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce31cc-4648-4566-9689-80ffb4b95a57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:48:42.363749 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6cce31cc-4648-4566-9689-80ffb4b95a57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:48:42.375869 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cce31cc-4648-4566-9689-80ffb4b95a57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:48:42.377396 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cce31cc-4648-4566-9689-80ffb4b95a57 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:48:42.388478 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1434/2872] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:48:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:48:48.277822 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfe865-fcba-464b-8b8c-dfc06a637389 service nova] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:48:48.294018 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfe865-fcba-464b-8b8c-dfc06a637389 service nova] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:48:48.303768 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f5fcb7-b0e4-4a90-95f9-a760a8e88795 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:48:48.304556 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5fcb7-b0e4-4a90-95f9-a760a8e88795 service nova] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:48:48.304890 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5fcb7-b0e4-4a90-95f9-a760a8e88795 service nova] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:48:48.305677 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5fcb7-b0e4-4a90-95f9-a760a8e88795 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:48:48.307565 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-51f5fcb7-b0e4-4a90-95f9-a760a8e88795 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:48:48.321269 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f5fcb7-b0e4-4a90-95f9-a760a8e88795 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:48:48.323019 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f5fcb7-b0e4-4a90-95f9-a760a8e88795 service nova] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:48:48.336803 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1439/2873] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:48:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:48:49.713950 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea893d-b2ec-42be-a2e9-d22ce98eff55 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:48:49.714489 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea893d-b2ec-42be-a2e9-d22ce98eff55 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:48:49.714595 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea893d-b2ec-42be-a2e9-d22ce98eff55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:48:49.732981 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-32ea893d-b2ec-42be-a2e9-d22ce98eff55 None None] MFA Rules not processed for user `67aa048d409c46e9a729aec9d1b2286e`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:48:49.734159 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-32ea893d-b2ec-42be-a2e9-d22ce98eff55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:48:49.741905 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1435/2874] 10.222.0.15 () {58 vars in 967 bytes} [Wed Jun 15 19:48:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:48:49.788285 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f5fcb7-b0e4-4a90-95f9-a760a8e88795 service nova] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:48:49.799864 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f5fcb7-b0e4-4a90-95f9-a760a8e88795 service nova] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:48:49.804929 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10189f25-b384-4f23-a8ac-77c119337279 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '176d5775a70f4524a9b7fe1ce9d426ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:48:49.805579 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10189f25-b384-4f23-a8ac-77c119337279 service swift] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:48:49.805861 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10189f25-b384-4f23-a8ac-77c119337279 service swift] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:48:49.806171 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10189f25-b384-4f23-a8ac-77c119337279 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:48:49.815154 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10189f25-b384-4f23-a8ac-77c119337279 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:48:49.816786 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10189f25-b384-4f23-a8ac-77c119337279 service swift] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:48:49.825863 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1440/2875] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Jun 15 19:48:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 19:48:52.991743 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea893d-b2ec-42be-a2e9-d22ce98eff55 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:48:52.993161 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-32ea893d-b2ec-42be-a2e9-d22ce98eff55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:48:53.085750 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea893d-b2ec-42be-a2e9-d22ce98eff55 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:48:53.091099 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb85eee4-511a-4448-97fc-d4ab88cbd87d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb0ec2ef6c4748e181f1963e7dfe30b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:48:53.091839 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb85eee4-511a-4448-97fc-d4ab88cbd87d service placement] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:48:53.092123 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb85eee4-511a-4448-97fc-d4ab88cbd87d service placement] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:48:53.092396 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb85eee4-511a-4448-97fc-d4ab88cbd87d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:48:53.094284 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-bb85eee4-511a-4448-97fc-d4ab88cbd87d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:48:53.107067 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb85eee4-511a-4448-97fc-d4ab88cbd87d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:48:53.108737 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb85eee4-511a-4448-97fc-d4ab88cbd87d service placement] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:48:53.122484 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1436/2876] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:48:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:48:56.286892 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10189f25-b384-4f23-a8ac-77c119337279 service swift] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:48:56.369495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10189f25-b384-4f23-a8ac-77c119337279 service swift] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:48:56.375342 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aff263-ce08-4abf-bbfa-dc5d8f0a4e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:48:56.376468 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aff263-ce08-4abf-bbfa-dc5d8f0a4e2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:48:56.376750 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aff263-ce08-4abf-bbfa-dc5d8f0a4e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:48:56.377030 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aff263-ce08-4abf-bbfa-dc5d8f0a4e2c admin admin] PATH_INFO: `/v3/users/841ccfe45fc64b15b97426ee692deb5b` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:48:56.378299 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aff263-ce08-4abf-bbfa-dc5d8f0a4e2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=841ccfe45fc64b15b97426ee692deb5b)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:48:56.380456 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aff263-ce08-4abf-bbfa-dc5d8f0a4e2c admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:48:56.431233 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-66aff263-ce08-4abf-bbfa-dc5d8f0a4e2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '841ccfe45fc64b15b97426ee692deb5b'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:48:56.431949 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-66aff263-ce08-4abf-bbfa-dc5d8f0a4e2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '841ccfe45fc64b15b97426ee692deb5b'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:48:56.439373 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-66aff263-ce08-4abf-bbfa-dc5d8f0a4e2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '841ccfe45fc64b15b97426ee692deb5b'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:48:56.440037 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-66aff263-ce08-4abf-bbfa-dc5d8f0a4e2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '841ccfe45fc64b15b97426ee692deb5b'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:48:56.448438 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-66aff263-ce08-4abf-bbfa-dc5d8f0a4e2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '841ccfe45fc64b15b97426ee692deb5b'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:48:56.467388 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1441/2877] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:48:56 2022] DELETE /identity/v3/users/841ccfe45fc64b15b97426ee692deb5b => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:48:56.646447 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb85eee4-511a-4448-97fc-d4ab88cbd87d service placement] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:48:56.658752 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb85eee4-511a-4448-97fc-d4ab88cbd87d service placement] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:48:56.719656 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431f00d-0b72-4218-8289-d2aca8ba545d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:48:56.720511 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431f00d-0b72-4218-8289-d2aca8ba545d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:48:56.720789 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431f00d-0b72-4218-8289-d2aca8ba545d admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:48:56.721081 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431f00d-0b72-4218-8289-d2aca8ba545d admin admin] PATH_INFO: `/v3/projects/5eeb6d09ac2a459183dac0e66a26b4bc` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:48:56.722310 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2431f00d-0b72-4218-8289-d2aca8ba545d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5eeb6d09ac2a459183dac0e66a26b4bc)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:48:56.724213 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2431f00d-0b72-4218-8289-d2aca8ba545d admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:48:56.772522 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2431f00d-0b72-4218-8289-d2aca8ba545d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eeb6d09ac2a459183dac0e66a26b4bc', 'request_id': 'req-2431f00d-0b72-4218-8289-d2aca8ba545d'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:48:56.781699 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2431f00d-0b72-4218-8289-d2aca8ba545d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eeb6d09ac2a459183dac0e66a26b4bc', 'request_id': 'req-2431f00d-0b72-4218-8289-d2aca8ba545d'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:48:56.788072 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2431f00d-0b72-4218-8289-d2aca8ba545d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eeb6d09ac2a459183dac0e66a26b4bc', 'request_id': 'req-2431f00d-0b72-4218-8289-d2aca8ba545d'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:48:56.795296 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:48:56.799918 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2431f00d-0b72-4218-8289-d2aca8ba545d admin admin] The token cache is being invalidate because project 5eeb6d09ac2a459183dac0e66a26b4bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:48:56.800292 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2431f00d-0b72-4218-8289-d2aca8ba545d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:48:56.801243 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2431f00d-0b72-4218-8289-d2aca8ba545d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:48:56.803350 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1437/2878] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:48:56 2022] DELETE /identity/v3/projects/5eeb6d09ac2a459183dac0e66a26b4bc => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:48:56.807844 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aff263-ce08-4abf-bbfa-dc5d8f0a4e2c admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:48:56.809257 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-66aff263-ce08-4abf-bbfa-dc5d8f0a4e2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:48:56.827197 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aff263-ce08-4abf-bbfa-dc5d8f0a4e2c admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:48:56.866394 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:48:56.888234 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da02b739-9a51-4339-9adf-4fc432acb94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:48:56.889148 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02b739-9a51-4339-9adf-4fc432acb94f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:48:56.889431 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02b739-9a51-4339-9adf-4fc432acb94f admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:48:56.889834 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02b739-9a51-4339-9adf-4fc432acb94f admin admin] PATH_INFO: `/v3/users/7e83ce09589645138e7ef934cbd98d5d` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:48:56.891228 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da02b739-9a51-4339-9adf-4fc432acb94f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e83ce09589645138e7ef934cbd98d5d)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:48:56.893180 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da02b739-9a51-4339-9adf-4fc432acb94f admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:48:56.941542 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-da02b739-9a51-4339-9adf-4fc432acb94f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e83ce09589645138e7ef934cbd98d5d'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:48:56.942232 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-da02b739-9a51-4339-9adf-4fc432acb94f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e83ce09589645138e7ef934cbd98d5d'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:48:56.951962 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-da02b739-9a51-4339-9adf-4fc432acb94f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e83ce09589645138e7ef934cbd98d5d'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:48:56.952657 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-da02b739-9a51-4339-9adf-4fc432acb94f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e83ce09589645138e7ef934cbd98d5d'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:48:56.960346 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-da02b739-9a51-4339-9adf-4fc432acb94f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e83ce09589645138e7ef934cbd98d5d'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:48:56.978587 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1442/2879] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:48:56 2022] DELETE /identity/v3/users/7e83ce09589645138e7ef934cbd98d5d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:48:57.162130 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431f00d-0b72-4218-8289-d2aca8ba545d admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:48:57.163595 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-2431f00d-0b72-4218-8289-d2aca8ba545d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:48:57.181119 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431f00d-0b72-4218-8289-d2aca8ba545d admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:48:57.248623 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a0da2-ac50-4ce2-8627-7e5ccd94bf66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:48:57.249491 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a0da2-ac50-4ce2-8627-7e5ccd94bf66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:48:57.249777 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a0da2-ac50-4ce2-8627-7e5ccd94bf66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:48:57.250060 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a0da2-ac50-4ce2-8627-7e5ccd94bf66 admin admin] PATH_INFO: `/v3/projects/5ad09f273d794c659d2c90c019b4aeeb` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:48:57.251310 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5a0da2-ac50-4ce2-8627-7e5ccd94bf66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ad09f273d794c659d2c90c019b4aeeb)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:48:57.253244 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5a0da2-ac50-4ce2-8627-7e5ccd94bf66 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:48:57.312756 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ac5a0da2-ac50-4ce2-8627-7e5ccd94bf66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ad09f273d794c659d2c90c019b4aeeb', 'request_id': 'req-ac5a0da2-ac50-4ce2-8627-7e5ccd94bf66'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:48:57.321060 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ac5a0da2-ac50-4ce2-8627-7e5ccd94bf66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ad09f273d794c659d2c90c019b4aeeb', 'request_id': 'req-ac5a0da2-ac50-4ce2-8627-7e5ccd94bf66'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:48:57.328025 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ac5a0da2-ac50-4ce2-8627-7e5ccd94bf66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ad09f273d794c659d2c90c019b4aeeb', 'request_id': 'req-ac5a0da2-ac50-4ce2-8627-7e5ccd94bf66'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:48:57.344327 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ac5a0da2-ac50-4ce2-8627-7e5ccd94bf66 admin admin] The token cache is being invalidate because project 5ad09f273d794c659d2c90c019b4aeeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:48:57.345078 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ac5a0da2-ac50-4ce2-8627-7e5ccd94bf66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:48:57.345780 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ac5a0da2-ac50-4ce2-8627-7e5ccd94bf66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:48:57.347430 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1438/2880] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:48:57 2022] DELETE /identity/v3/projects/5ad09f273d794c659d2c90c019b4aeeb => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:49:06.974498 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=35,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:49:07.336245 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:49:11.123123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da02b739-9a51-4339-9adf-4fc432acb94f admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:49:11.123123 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-da02b739-9a51-4339-9adf-4fc432acb94f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:49:11.186787 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da02b739-9a51-4339-9adf-4fc432acb94f admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:49:11.249473 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02512a9f-9534-4c15-8151-8dc69d03c564 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:49:11.250180 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02512a9f-9534-4c15-8151-8dc69d03c564 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:49:11.250468 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02512a9f-9534-4c15-8151-8dc69d03c564 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:49:11.250746 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02512a9f-9534-4c15-8151-8dc69d03c564 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:49:11.252677 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-02512a9f-9534-4c15-8151-8dc69d03c564 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:49:11.356539 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02512a9f-9534-4c15-8151-8dc69d03c564 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:49:11.357997 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02512a9f-9534-4c15-8151-8dc69d03c564 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:49:11.369041 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1443/2881] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:49:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:49:16.498548 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a0da2-ac50-4ce2-8627-7e5ccd94bf66 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:49:16.500043 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ac5a0da2-ac50-4ce2-8627-7e5ccd94bf66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:49:16.514389 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a0da2-ac50-4ce2-8627-7e5ccd94bf66 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:49:16.519237 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5692741a-480e-434e-bcb8-4884acd6d2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:49:16.520115 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692741a-480e-434e-bcb8-4884acd6d2cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:49:16.520399 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692741a-480e-434e-bcb8-4884acd6d2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:49:16.520671 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692741a-480e-434e-bcb8-4884acd6d2cb admin admin] PATH_INFO: `/v3/users/4ca9a3e7826b4928863bd83a9496205a` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:49:16.521876 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5692741a-480e-434e-bcb8-4884acd6d2cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ca9a3e7826b4928863bd83a9496205a)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:49:16.523906 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5692741a-480e-434e-bcb8-4884acd6d2cb admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:49:16.585067 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5692741a-480e-434e-bcb8-4884acd6d2cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ca9a3e7826b4928863bd83a9496205a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:49:16.585764 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5692741a-480e-434e-bcb8-4884acd6d2cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ca9a3e7826b4928863bd83a9496205a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:49:16.586404 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5692741a-480e-434e-bcb8-4884acd6d2cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ca9a3e7826b4928863bd83a9496205a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:49:16.600039 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5692741a-480e-434e-bcb8-4884acd6d2cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ca9a3e7826b4928863bd83a9496205a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:49:16.610263 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5692741a-480e-434e-bcb8-4884acd6d2cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ca9a3e7826b4928863bd83a9496205a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:49:16.625045 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1439/2882] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:49:16 2022] DELETE /identity/v3/users/4ca9a3e7826b4928863bd83a9496205a => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:49:16.852469 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02512a9f-9534-4c15-8151-8dc69d03c564 service neutron] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:49:16.865750 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02512a9f-9534-4c15-8151-8dc69d03c564 service neutron] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:49:16.933450 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a942a-0ca1-44e9-8d20-fc3359ca20bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:49:16.935661 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a942a-0ca1-44e9-8d20-fc3359ca20bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:49:16.935955 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a942a-0ca1-44e9-8d20-fc3359ca20bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:49:16.936240 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a942a-0ca1-44e9-8d20-fc3359ca20bb admin admin] PATH_INFO: `/v3/projects/feaa96a8901e42c18c270c092a40fc83` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:49:16.937732 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564a942a-0ca1-44e9-8d20-fc3359ca20bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=feaa96a8901e42c18c270c092a40fc83)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:49:16.939706 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564a942a-0ca1-44e9-8d20-fc3359ca20bb admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:49:16.949152 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=100 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:49:16.993988 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-564a942a-0ca1-44e9-8d20-fc3359ca20bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'feaa96a8901e42c18c270c092a40fc83', 'request_id': 'req-564a942a-0ca1-44e9-8d20-fc3359ca20bb'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:49:17.001966 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-564a942a-0ca1-44e9-8d20-fc3359ca20bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'feaa96a8901e42c18c270c092a40fc83', 'request_id': 'req-564a942a-0ca1-44e9-8d20-fc3359ca20bb'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:49:17.009397 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-564a942a-0ca1-44e9-8d20-fc3359ca20bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'feaa96a8901e42c18c270c092a40fc83', 'request_id': 'req-564a942a-0ca1-44e9-8d20-fc3359ca20bb'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:49:17.020319 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-564a942a-0ca1-44e9-8d20-fc3359ca20bb admin admin] The token cache is being invalidate because project feaa96a8901e42c18c270c092a40fc83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:49:17.020610 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-564a942a-0ca1-44e9-8d20-fc3359ca20bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:49:17.021281 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-564a942a-0ca1-44e9-8d20-fc3359ca20bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:49:17.022925 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1444/2883] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:49:16 2022] DELETE /identity/v3/projects/feaa96a8901e42c18c270c092a40fc83 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:49:17.026987 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5692741a-480e-434e-bcb8-4884acd6d2cb admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:49:17.028402 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-5692741a-480e-434e-bcb8-4884acd6d2cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:49:17.042907 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5692741a-480e-434e-bcb8-4884acd6d2cb admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:49:17.102993 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3429ae5-9b74-471f-9bfd-5445abf1b01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:49:17.103786 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3429ae5-9b74-471f-9bfd-5445abf1b01f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:49:17.104065 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3429ae5-9b74-471f-9bfd-5445abf1b01f admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:49:17.104338 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3429ae5-9b74-471f-9bfd-5445abf1b01f admin admin] PATH_INFO: `/v3/users/8ce2eee120f846aa8c552cde8469d994` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:49:17.105543 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3429ae5-9b74-471f-9bfd-5445abf1b01f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ce2eee120f846aa8c552cde8469d994)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:49:17.107538 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3429ae5-9b74-471f-9bfd-5445abf1b01f admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:49:17.154189 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-d3429ae5-9b74-471f-9bfd-5445abf1b01f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ce2eee120f846aa8c552cde8469d994'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:49:17.154842 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-d3429ae5-9b74-471f-9bfd-5445abf1b01f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ce2eee120f846aa8c552cde8469d994'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:49:17.155441 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-d3429ae5-9b74-471f-9bfd-5445abf1b01f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ce2eee120f846aa8c552cde8469d994'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:49:17.168280 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-d3429ae5-9b74-471f-9bfd-5445abf1b01f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ce2eee120f846aa8c552cde8469d994'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:49:17.175246 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-d3429ae5-9b74-471f-9bfd-5445abf1b01f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ce2eee120f846aa8c552cde8469d994'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:49:17.176901 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:49:17.189771 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1440/2884] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:49:17 2022] DELETE /identity/v3/users/8ce2eee120f846aa8c552cde8469d994 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:49:17.372132 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a942a-0ca1-44e9-8d20-fc3359ca20bb admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:49:17.373589 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-564a942a-0ca1-44e9-8d20-fc3359ca20bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:49:17.386689 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a942a-0ca1-44e9-8d20-fc3359ca20bb admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:49:17.446115 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f4580-0b5c-401a-8387-8129a9f214c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:49:17.446992 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f4580-0b5c-401a-8387-8129a9f214c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:49:17.447273 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f4580-0b5c-401a-8387-8129a9f214c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:49:17.447546 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f4580-0b5c-401a-8387-8129a9f214c4 admin admin] PATH_INFO: `/v3/projects/2f7874ef29f7408388914096a7b56893` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:49:17.448733 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f4580-0b5c-401a-8387-8129a9f214c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f7874ef29f7408388914096a7b56893)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:49:17.450684 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f4580-0b5c-401a-8387-8129a9f214c4 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:49:17.501250 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-a68f4580-0b5c-401a-8387-8129a9f214c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f7874ef29f7408388914096a7b56893', 'request_id': 'req-a68f4580-0b5c-401a-8387-8129a9f214c4'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:49:17.510553 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-a68f4580-0b5c-401a-8387-8129a9f214c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f7874ef29f7408388914096a7b56893', 'request_id': 'req-a68f4580-0b5c-401a-8387-8129a9f214c4'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:49:17.517350 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-a68f4580-0b5c-401a-8387-8129a9f214c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f7874ef29f7408388914096a7b56893', 'request_id': 'req-a68f4580-0b5c-401a-8387-8129a9f214c4'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:49:17.529418 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-a68f4580-0b5c-401a-8387-8129a9f214c4 admin admin] The token cache is being invalidate because project 2f7874ef29f7408388914096a7b56893 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:49:17.529714 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-a68f4580-0b5c-401a-8387-8129a9f214c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:49:17.530473 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-a68f4580-0b5c-401a-8387-8129a9f214c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:49:17.532083 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1445/2885] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:49:17 2022] DELETE /identity/v3/projects/2f7874ef29f7408388914096a7b56893 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:49:27.189020 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:49:27.526081 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=72,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:49:33.093040 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe9377-16ab-4e54-b411-16c264f4b210 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:49:33.093654 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe9377-16ab-4e54-b411-16c264f4b210 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:49:33.093654 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe9377-16ab-4e54-b411-16c264f4b210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:49:33.111632 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-edfe9377-16ab-4e54-b411-16c264f4b210 None None] MFA Rules not processed for user `f3eb282e55c941ef9fefb57e14a708f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:49:33.112857 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-edfe9377-16ab-4e54-b411-16c264f4b210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:49:33.169192 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1441/2886] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:49:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:49:35.967027 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f4580-0b5c-401a-8387-8129a9f214c4 admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:49:35.968471 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-a68f4580-0b5c-401a-8387-8129a9f214c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:49:36.055185 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f4580-0b5c-401a-8387-8129a9f214c4 admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:49:36.060268 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8677d2-965a-44ab-b2d7-ec0f49d38387 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb0ec2ef6c4748e181f1963e7dfe30b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:49:36.060941 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8677d2-965a-44ab-b2d7-ec0f49d38387 service placement] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:49:36.061237 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8677d2-965a-44ab-b2d7-ec0f49d38387 service placement] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:49:36.061523 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8677d2-965a-44ab-b2d7-ec0f49d38387 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:49:36.063440 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-bb8677d2-965a-44ab-b2d7-ec0f49d38387 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:49:36.076927 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8677d2-965a-44ab-b2d7-ec0f49d38387 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:49:36.078573 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8677d2-965a-44ab-b2d7-ec0f49d38387 service placement] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:49:36.089528 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1446/2887] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:49:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:49:43.162053 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=25 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:49:46.085965 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=32 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:49:53.266381 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe9377-16ab-4e54-b411-16c264f4b210 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:49:53.267830 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-edfe9377-16ab-4e54-b411-16c264f4b210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:49:53.372270 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe9377-16ab-4e54-b411-16c264f4b210 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:49:53.378922 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c9a7b-3dde-45ab-8094-432710b7ac5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:49:53.379369 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c9a7b-3dde-45ab-8094-432710b7ac5a service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:49:53.379651 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c9a7b-3dde-45ab-8094-432710b7ac5a service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:49:53.379925 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c9a7b-3dde-45ab-8094-432710b7ac5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:49:53.381759 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b12c9a7b-3dde-45ab-8094-432710b7ac5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:49:53.396588 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12c9a7b-3dde-45ab-8094-432710b7ac5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:49:53.398306 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12c9a7b-3dde-45ab-8094-432710b7ac5a service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:49:53.410206 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1442/2888] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:49:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:50:03.405879 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=34 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:51:31.975544 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb497bcd-aea0-4c69-99c9-d13da07b1ee3 None None] REQUEST_METHOD: `POST` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:51:31.976066 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb497bcd-aea0-4c69-99c9-d13da07b1ee3 None None] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:51:31.976188 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb497bcd-aea0-4c69-99c9-d13da07b1ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:51:32.002879 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.auth.core [None req-eb497bcd-aea0-4c69-99c9-d13da07b1ee3 None None] MFA Rules not processed for user `f3eb282e55c941ef9fefb57e14a708f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80223) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 19:51:32.004073 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-eb497bcd-aea0-4c69-99c9-d13da07b1ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:51:32.027907 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1447/2889] 10.222.0.15 () {58 vars in 957 bytes} [Wed Jun 15 19:51:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 19:51:33.439780 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c9a7b-3dde-45ab-8094-432710b7ac5a service nova] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:51:33.444163 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b12c9a7b-3dde-45ab-8094-432710b7ac5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:51:33.530754 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c9a7b-3dde-45ab-8094-432710b7ac5a service nova] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:51:33.535876 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be5a957-457f-4d4a-9523-26f4b2ebec25 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb0ec2ef6c4748e181f1963e7dfe30b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:51:33.536533 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5a957-457f-4d4a-9523-26f4b2ebec25 service placement] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:51:33.536815 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5a957-457f-4d4a-9523-26f4b2ebec25 service placement] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:51:33.537093 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5a957-457f-4d4a-9523-26f4b2ebec25 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:51:33.538938 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3be5a957-457f-4d4a-9523-26f4b2ebec25 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:51:33.553618 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be5a957-457f-4d4a-9523-26f4b2ebec25 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:51:33.555237 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be5a957-457f-4d4a-9523-26f4b2ebec25 service placement] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:51:33.565284 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1443/2890] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:51:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:51:42.025846 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=15 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:51:43.560908 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=37 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:51:44.574417 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb497bcd-aea0-4c69-99c9-d13da07b1ee3 None None] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:51:44.605377 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb497bcd-aea0-4c69-99c9-d13da07b1ee3 None None] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:51:44.610657 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25d92e-219d-4c93-a0a8-e3bb60fb1c26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e74ac44e5814122a3226ac023f3688a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:51:44.611299 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25d92e-219d-4c93-a0a8-e3bb60fb1c26 service nova] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:51:44.611582 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25d92e-219d-4c93-a0a8-e3bb60fb1c26 service nova] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:51:44.611858 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25d92e-219d-4c93-a0a8-e3bb60fb1c26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:51:44.613596 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-ea25d92e-219d-4c93-a0a8-e3bb60fb1c26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:51:44.624665 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea25d92e-219d-4c93-a0a8-e3bb60fb1c26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:51:44.626252 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea25d92e-219d-4c93-a0a8-e3bb60fb1c26 service nova] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:51:44.636340 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1448/2891] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:51:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:51:54.631422 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=13 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:53:30.312965 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be5a957-457f-4d4a-9523-26f4b2ebec25 service placement] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:53:30.315038 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3be5a957-457f-4d4a-9523-26f4b2ebec25 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:53:30.410383 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be5a957-457f-4d4a-9523-26f4b2ebec25 service placement] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:53:30.414878 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ddf716-5a2b-4c07-bdc1-3f4908917ff7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:53:30.415790 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ddf716-5a2b-4c07-bdc1-3f4908917ff7 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:53:30.416086 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ddf716-5a2b-4c07-bdc1-3f4908917ff7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:53:30.416361 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ddf716-5a2b-4c07-bdc1-3f4908917ff7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:53:30.418239 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e6ddf716-5a2b-4c07-bdc1-3f4908917ff7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:53:30.448483 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ddf716-5a2b-4c07-bdc1-3f4908917ff7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:53:30.449943 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ddf716-5a2b-4c07-bdc1-3f4908917ff7 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:53:30.462235 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1444/2892] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:53:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:53:31.644527 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25d92e-219d-4c93-a0a8-e3bb60fb1c26 service nova] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:53:31.654888 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25d92e-219d-4c93-a0a8-e3bb60fb1c26 service nova] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:53:31.658816 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836af4f1-5eca-4df8-ba5e-be50a517be81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:53:31.659312 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836af4f1-5eca-4df8-ba5e-be50a517be81 service neutron] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:53:31.659601 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836af4f1-5eca-4df8-ba5e-be50a517be81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:53:31.659881 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836af4f1-5eca-4df8-ba5e-be50a517be81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:53:31.661691 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-836af4f1-5eca-4df8-ba5e-be50a517be81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:53:31.676096 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836af4f1-5eca-4df8-ba5e-be50a517be81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:53:31.677668 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836af4f1-5eca-4df8-ba5e-be50a517be81 service neutron] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:53:31.690259 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1449/2893] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:53:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:53:40.456304 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=40 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:53:41.684889 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=6 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:53:42.870990 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ddf716-5a2b-4c07-bdc1-3f4908917ff7 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:53:42.872793 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e6ddf716-5a2b-4c07-bdc1-3f4908917ff7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:53:42.884699 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ddf716-5a2b-4c07-bdc1-3f4908917ff7 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:53:42.888541 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd3256-5910-4b9b-b1d1-ba11b8904012 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c63e0b6ceed4f7cb2e6f9269eb85795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:53:42.889179 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd3256-5910-4b9b-b1d1-ba11b8904012 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:53:42.889459 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd3256-5910-4b9b-b1d1-ba11b8904012 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:53:42.889732 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd3256-5910-4b9b-b1d1-ba11b8904012 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:53:42.891646 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c6fd3256-5910-4b9b-b1d1-ba11b8904012 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:53:42.979958 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fd3256-5910-4b9b-b1d1-ba11b8904012 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:53:42.981394 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fd3256-5910-4b9b-b1d1-ba11b8904012 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:53:43.001641 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1445/2894] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:53:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:53:46.162206 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836af4f1-5eca-4df8-ba5e-be50a517be81 service neutron] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:53:46.191021 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836af4f1-5eca-4df8-ba5e-be50a517be81 service neutron] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:53:46.197488 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2731133-dfde-42b1-90f6-234d73a2ae4d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb0ec2ef6c4748e181f1963e7dfe30b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:53:46.198656 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2731133-dfde-42b1-90f6-234d73a2ae4d service placement] REQUEST_METHOD: `GET` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:53:46.198943 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2731133-dfde-42b1-90f6-234d73a2ae4d service placement] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:53:46.199219 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2731133-dfde-42b1-90f6-234d73a2ae4d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:53:46.201351 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-b2731133-dfde-42b1-90f6-234d73a2ae4d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:53:46.230484 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2731133-dfde-42b1-90f6-234d73a2ae4d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:53:46.232364 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2731133-dfde-42b1-90f6-234d73a2ae4d service placement] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:53:46.261699 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1450/2895] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:53:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:53:49.531941 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd3256-5910-4b9b-b1d1-ba11b8904012 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:53:49.533519 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c6fd3256-5910-4b9b-b1d1-ba11b8904012 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:53:49.549639 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd3256-5910-4b9b-b1d1-ba11b8904012 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:53:49.554854 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd036772-dcfd-4b3c-9e1c-58eff61e05b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:53:49.555729 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd036772-dcfd-4b3c-9e1c-58eff61e05b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:53:49.556034 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd036772-dcfd-4b3c-9e1c-58eff61e05b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:53:49.556310 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd036772-dcfd-4b3c-9e1c-58eff61e05b8 admin admin] PATH_INFO: `/v3/users/f3eb282e55c941ef9fefb57e14a708f1` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:53:49.557490 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd036772-dcfd-4b3c-9e1c-58eff61e05b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3eb282e55c941ef9fefb57e14a708f1)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:53:49.559471 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd036772-dcfd-4b3c-9e1c-58eff61e05b8 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:53:49.610530 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-fd036772-dcfd-4b3c-9e1c-58eff61e05b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3eb282e55c941ef9fefb57e14a708f1'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:53:49.611273 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-fd036772-dcfd-4b3c-9e1c-58eff61e05b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3eb282e55c941ef9fefb57e14a708f1'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:53:49.612012 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-fd036772-dcfd-4b3c-9e1c-58eff61e05b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3eb282e55c941ef9fefb57e14a708f1'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:53:49.627580 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-fd036772-dcfd-4b3c-9e1c-58eff61e05b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3eb282e55c941ef9fefb57e14a708f1'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:53:49.636030 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-fd036772-dcfd-4b3c-9e1c-58eff61e05b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3eb282e55c941ef9fefb57e14a708f1'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:53:49.650370 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1446/2896] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:53:49 2022] DELETE /identity/v3/users/f3eb282e55c941ef9fefb57e14a708f1 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:53:49.840145 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2731133-dfde-42b1-90f6-234d73a2ae4d service placement] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:53:49.852495 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2731133-dfde-42b1-90f6-234d73a2ae4d service placement] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:53:49.930056 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b1c46-5b4f-4ca7-8850-5d4b759fcc8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:53:49.930978 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b1c46-5b4f-4ca7-8850-5d4b759fcc8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:53:49.931256 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b1c46-5b4f-4ca7-8850-5d4b759fcc8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:53:49.931530 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b1c46-5b4f-4ca7-8850-5d4b759fcc8e admin admin] PATH_INFO: `/v3/projects/b7cdb8d3ddc84d0fbd6128c3b46d05f7` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:53:49.932757 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963b1c46-5b4f-4ca7-8850-5d4b759fcc8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7cdb8d3ddc84d0fbd6128c3b46d05f7)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:53:49.934708 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963b1c46-5b4f-4ca7-8850-5d4b759fcc8e admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:53:49.986208 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-963b1c46-5b4f-4ca7-8850-5d4b759fcc8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7cdb8d3ddc84d0fbd6128c3b46d05f7', 'request_id': 'req-963b1c46-5b4f-4ca7-8850-5d4b759fcc8e'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:53:49.993206 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-963b1c46-5b4f-4ca7-8850-5d4b759fcc8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7cdb8d3ddc84d0fbd6128c3b46d05f7', 'request_id': 'req-963b1c46-5b4f-4ca7-8850-5d4b759fcc8e'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:53:49.999714 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-963b1c46-5b4f-4ca7-8850-5d4b759fcc8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7cdb8d3ddc84d0fbd6128c3b46d05f7', 'request_id': 'req-963b1c46-5b4f-4ca7-8850-5d4b759fcc8e'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:53:50.010418 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-963b1c46-5b4f-4ca7-8850-5d4b759fcc8e admin admin] The token cache is being invalidate because project b7cdb8d3ddc84d0fbd6128c3b46d05f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:53:50.010704 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-963b1c46-5b4f-4ca7-8850-5d4b759fcc8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:53:50.011334 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-963b1c46-5b4f-4ca7-8850-5d4b759fcc8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:53:50.012931 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1451/2897] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:53:49 2022] DELETE /identity/v3/projects/b7cdb8d3ddc84d0fbd6128c3b46d05f7 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:53:50.017032 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd036772-dcfd-4b3c-9e1c-58eff61e05b8 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:53:50.018476 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-fd036772-dcfd-4b3c-9e1c-58eff61e05b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:53:50.031753 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd036772-dcfd-4b3c-9e1c-58eff61e05b8 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:53:50.064974 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:53:50.089838 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb546cc4-424a-4942-b0cd-a70bba73774d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:53:50.090752 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb546cc4-424a-4942-b0cd-a70bba73774d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:53:50.091041 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb546cc4-424a-4942-b0cd-a70bba73774d admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:53:50.091345 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb546cc4-424a-4942-b0cd-a70bba73774d admin admin] PATH_INFO: `/v3/users/3c085f1659164832949fe79b67b88625` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:53:50.092624 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb546cc4-424a-4942-b0cd-a70bba73774d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c085f1659164832949fe79b67b88625)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:53:50.094771 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb546cc4-424a-4942-b0cd-a70bba73774d admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:53:50.143258 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-eb546cc4-424a-4942-b0cd-a70bba73774d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c085f1659164832949fe79b67b88625'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:53:50.143905 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-eb546cc4-424a-4942-b0cd-a70bba73774d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c085f1659164832949fe79b67b88625'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:53:50.144656 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-eb546cc4-424a-4942-b0cd-a70bba73774d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c085f1659164832949fe79b67b88625'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:53:50.157043 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-eb546cc4-424a-4942-b0cd-a70bba73774d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c085f1659164832949fe79b67b88625'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:53:50.164692 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-eb546cc4-424a-4942-b0cd-a70bba73774d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c085f1659164832949fe79b67b88625'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:53:50.177415 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1447/2898] 10.222.0.15 () {58 vars in 1223 bytes} [Wed Jun 15 19:53:50 2022] DELETE /identity/v3/users/3c085f1659164832949fe79b67b88625 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:53:50.357001 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b1c46-5b4f-4ca7-8850-5d4b759fcc8e admin admin] Authenticating user token {{(pid=80223) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:53:50.358541 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.fernet_utils [None req-963b1c46-5b4f-4ca7-8850-5d4b759fcc8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80223) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:53:50.371631 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b1c46-5b4f-4ca7-8850-5d4b759fcc8e admin admin] Validating token access rules against request {{(pid=80223) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:53:50.432601 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec67667-502e-49d4-a8e1-b82fcca357c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd018cc34dbeb4d56a3052de5c20b961c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d07be90a27c40718a346a1d9e5469b4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80223) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:53:50.433559 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec67667-502e-49d4-a8e1-b82fcca357c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:53:50.433844 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec67667-502e-49d4-a8e1-b82fcca357c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:53:50.434148 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec67667-502e-49d4-a8e1-b82fcca357c4 admin admin] PATH_INFO: `/v3/projects/f36196d5f3d049eb8cd537ff68322c00` {{(pid=80223) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:53:50.435346 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec67667-502e-49d4-a8e1-b82fcca357c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f36196d5f3d049eb8cd537ff68322c00)` {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:53:50.437265 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec67667-502e-49d4-a8e1-b82fcca357c4 admin admin] RBAC: Authorization granted {{(pid=80223) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:53:50.458875 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:53:50.485301 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-0ec67667-502e-49d4-a8e1-b82fcca357c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f36196d5f3d049eb8cd537ff68322c00', 'request_id': 'req-0ec67667-502e-49d4-a8e1-b82fcca357c4'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:53:50.492272 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-0ec67667-502e-49d4-a8e1-b82fcca357c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f36196d5f3d049eb8cd537ff68322c00', 'request_id': 'req-0ec67667-502e-49d4-a8e1-b82fcca357c4'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:53:50.499039 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-0ec67667-502e-49d4-a8e1-b82fcca357c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f36196d5f3d049eb8cd537ff68322c00', 'request_id': 'req-0ec67667-502e-49d4-a8e1-b82fcca357c4'} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:53:50.509941 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-0ec67667-502e-49d4-a8e1-b82fcca357c4 admin admin] The token cache is being invalidate because project f36196d5f3d049eb8cd537ff68322c00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80223) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 19:53:50.510270 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-0ec67667-502e-49d4-a8e1-b82fcca357c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:53:50.510934 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG keystone.notifications [None req-0ec67667-502e-49d4-a8e1-b82fcca357c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80223) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 19:53:50.512490 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: [pid: 80223|app: 0|req: 1452/2899] 10.222.0.15 () {58 vars in 1232 bytes} [Wed Jun 15 19:53:50 2022] DELETE /identity/v3/projects/f36196d5f3d049eb8cd537ff68322c00 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 19:54:00.174018 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=36,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:54:00.506345 dv-cinder-844563-3-iscsi devstack@keystone.service[80223]: DEBUG dbcounter [-] [80223] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80223) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 19:57:05.207948 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb546cc4-424a-4942-b0cd-a70bba73774d admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 19:57:05.209449 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-eb546cc4-424a-4942-b0cd-a70bba73774d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:57:05.281552 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb546cc4-424a-4942-b0cd-a70bba73774d admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 19:57:05.342876 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5615af-a8ce-4adb-9f6a-872ebe3d163f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb0ec2ef6c4748e181f1963e7dfe30b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb8cc0eb560465e972ffd45867ee73b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 19:57:05.343280 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5615af-a8ce-4adb-9f6a-872ebe3d163f service placement] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 19:57:05.343675 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5615af-a8ce-4adb-9f6a-872ebe3d163f service placement] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 19:57:05.344062 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5615af-a8ce-4adb-9f6a-872ebe3d163f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 19:57:05.346029 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3b5615af-a8ce-4adb-9f6a-872ebe3d163f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 19:57:05.357874 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5615af-a8ce-4adb-9f6a-872ebe3d163f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 19:57:05.359501 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5615af-a8ce-4adb-9f6a-872ebe3d163f service placement] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 19:57:05.370300 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1448/2900] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Jun 15 19:57:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 19:57:15.365580 dv-cinder-844563-3-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=58 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}